-- Logs begin at Sun 2022-08-07 23:33:38 UTC, end at Mon 2022-08-08 00:51:12 UTC. -- Aug 08 00:12:53.589749 dv-cinder-851640-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 00:12:53.597881 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 00:12:53 2022] *** Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: nodename: dv-cinder-851640-3-iscsi Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: machine: x86_64 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: clock source: unix Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: pcre jit disabled Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: detected number of CPU cores: 4 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: current working directory: / Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: detected binary path: /usr/bin/uwsgi-core Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: your processes number limit is 47680 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: your memory page size is 4096 bytes Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: detected max file descriptor number: 2048 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: lock engine: pthread robust mutexes Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: thunder lock: enabled Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 00:12:53.603247 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 00:12:53.640166 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: Python main interpreter initialized at 0x5572fc0a69c0 Aug 08 00:12:53.640166 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: python threads support enabled Aug 08 00:12:53.640166 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: your server socket listen backlog is limited to 100 connections Aug 08 00:12:53.640166 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: your mercy for graceful operations on workers is 90 seconds Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: *** Operational MODE: preforking *** Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: *** uWSGI is running in multiple interpreter mode *** Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: spawned uWSGI master process (pid: 91975) Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: spawned uWSGI worker 1 (pid: 91987, cores: 1) Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: spawned uWSGI worker 2 (pid: 91988, cores: 1) Aug 08 00:12:53.644066 dv-cinder-851640-3-iscsi devstack@keystone.service[91975]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 00:12:53.641052 dv-cinder-851640-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 00:12:59.175670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 00:12:59.175670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: from cryptography.utils import int_from_bytes Aug 08 00:12:59.176483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 00:12:59.176483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: from cryptography.utils import int_from_bytes Aug 08 00:12:59.195985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 00:12:59.195985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: from cryptography.utils import int_from_bytes Aug 08 00:12:59.196594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 00:12:59.196594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: from cryptography.utils import int_from_bytes Aug 08 00:12:59.226720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 00:12:59.226911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 00:12:59.227104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] command line args: [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 00:12:59.227403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 00:12:59.227581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ================================================================================ {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 00:12:59.227833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] admin_token = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.228075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] config_dir = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.228264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.228444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] config_source = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.228648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] debug = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.228821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.229322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.229414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.229616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.229864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.230092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.230270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_config_append = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.230450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.231173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_dir = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.231372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_file = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.231609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.231874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.232111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.232372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.232551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.232730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.232974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.233219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.233451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.233682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.233912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.234143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.234405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.234641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.234907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.235414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.235594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] publish_errors = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.235825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.236054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.236231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.236476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.236708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.236940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] standard_threads = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.237168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.237409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.237584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.237758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] use_journal = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.237932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] use_json = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.238106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] use_stderr = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.238282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] use_syslog = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.238509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.238761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.238996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.239278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.239521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.239770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.240007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.240254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.240490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.240736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.240984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.241234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.241469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.241719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.241951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.242196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.242443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.242679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.242932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.243185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.243450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.243683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.243935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.244193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.244272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 00:12:59.244444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 00:12:59.244530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.244635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] command line args: [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 00:12:59.244714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.244907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.244992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 00:12:59.245115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ================================================================================ {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 00:12:59.245193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.245351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.245427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] admin_token = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.245583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.245721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] config_dir = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.245828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.245957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.246079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.246770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] config_source = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.246908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.246993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] debug = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.247119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.247199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.247386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.247471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.247628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.247716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.247864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.247973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.248130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.248297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.248375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.248552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.248633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.248783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_config_append = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.248873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.249006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.249097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.249209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_dir = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.249294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.249431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_file = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.249517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.249703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.249781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.249937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.250044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.250183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.250356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.250435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.250632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.250714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.250821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.250904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.250987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.251081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.251289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.251373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.251577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.251668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.251807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.251884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.252041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.252145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.252276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.252442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.252520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.252728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.252806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.252961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.253047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.253189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.253332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.253436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.253668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.253756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.253918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.254165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.254331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.254418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.254532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] publish_errors = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.254646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.254831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.254915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.255101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.255186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.255353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.255441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.255626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.255704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.255863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.255941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.256115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.256365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.256534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] standard_threads = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.256613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.256830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.256907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.257064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.257182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.257312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.257415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.257537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] use_journal = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.257639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.257734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] use_json = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.257846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.257944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] use_stderr = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.258039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.258117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] use_syslog = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.258274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.258416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 00:12:59.258505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.258676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.258754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.258910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.259892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.260905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.261054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.261283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.261368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.261517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.261665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.261778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262886 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.262994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.263996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.external = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.264916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.password = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] auth.token = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.265932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.266882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.267842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.268942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.269934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.270923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.271921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.272954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.273906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.274996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.275946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.276997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.277131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.277210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.277408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.277543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.277681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.277893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.278945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.279896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.280958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.281975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.282895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.283875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.external = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.284988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.password = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] auth.token = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.285995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.286921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.287891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.288993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.289934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.290984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.291955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.292882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.293561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.293652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.293797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.293878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.294997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.295967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.296984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.297910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] role.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] role.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.298870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.299991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300218 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.300940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.301972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.302880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.303950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.304937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.305958 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.306878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.307853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.308008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.308094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.308251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.308329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.308483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.308562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.309987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.310958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.311944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.connection = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.312873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.313035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.313121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.313524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.313609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.313767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.313846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.314941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.315984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.316929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.317068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91987) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 00:12:59.317164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.317302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.317549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.317780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.318025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.318258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.318494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.318724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] role.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.318967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] role.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.319193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.319462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.319707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.319952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.320195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.320440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.320684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.320928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.321172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.321400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.321642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.321887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.322132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.322377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.322781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.323017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.323286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.323536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.323785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.324030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.324303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.324550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.324921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.325171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.325412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.325645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.325878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.326130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.326387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.326633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.326868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.327104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.327363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.327598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.327832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.328080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.328322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.328579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.328824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.329057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.329298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.329539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.329785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.330018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.330253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.330485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.330728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.330974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.331201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.331464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.331754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.332013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.connection = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.332258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.332503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.332738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.332976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.333208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.333441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.333675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.333908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.334149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.334383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.334623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.334854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.335098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.335358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.335598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.335825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.336063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.336293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 00:12:59.336461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91988) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 00:12:59.346054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.346290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.346517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.356891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.361395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.361627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.361849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.371987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.385383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.385614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.390291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.390521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.438171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.438540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.489168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.489303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.489399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.489482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.552883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.553966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.554049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.554187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.554404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.578799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.578909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.579954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.580037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.580164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.580246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.591007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.591423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.603614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.603840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.603924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.604920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91988) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.605002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.605133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91987) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 00:12:59.622073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.622288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 00:12:59.622622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.622823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 00:12:59.623041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.623140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.623286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 00:12:59.623406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.623619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.623712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.623817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 00:12:59.623926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.624092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.624353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.624612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.624868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.635027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.635325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.635594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.635873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.636235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.636513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.637436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.637719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.643396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.646925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.647208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.647544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.647805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.648172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.648436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.648704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.649030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.649243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.649707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.650037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.650885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.654179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.654467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.654755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.655044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.655357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.655628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.655928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.656209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.656485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.656763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.657034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.673953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.675474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.675769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.680368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.680460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.680653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.680931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.681890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.682182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.687486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.687779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.687986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.688072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.688573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.692527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.692816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.693097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.693382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.693659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.693931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.694197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.694462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.694955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.695266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.700094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.700597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.701002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.701293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.701586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.701869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.702145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.702449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.710616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.710894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.714786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.718174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.718356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.718529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.718639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.722579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.722817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.725745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.725993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.726077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.729138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.729439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.729738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.730032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.730313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.730550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.730638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.733786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.736986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.737293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.737583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.737874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.738152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.738446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.747261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.747557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.747823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.748082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.754807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.754900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.755114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.755456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.755758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.756426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.756704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.759518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.759801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.760097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.760389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.760666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.763178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.764757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.765038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.767861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.768150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.768455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.768759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.769042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.773007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.773292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.773569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.773848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.774128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.774400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.774673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.780446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.780732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.781015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.781300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.781585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.781875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.782163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.791358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.794427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.797351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.798889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.798978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.800409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.800702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.801002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.801872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.804840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.806412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.807358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.807970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.808265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.809053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.809497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.812438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.812727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.813011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.813532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.816039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.817657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.820623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.820922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.821229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.821502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.821588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.821794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.822073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.829512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=91988) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.829795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.830075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.830350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.830436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.830622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.830707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.830893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.830981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.831164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.831465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.831846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.832120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.832393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.832670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.832938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.833198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91988) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.837769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=91987) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 00:12:59.838057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.838342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.838619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.838911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.839191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.839501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.839772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.840221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.840507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.840788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.841065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.841338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.841612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91987) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 00:12:59.866805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572fc0a69c0 pid: 91988 (default app) Aug 08 00:12:59.870043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faac8b-d97c-424b-a049-f29a59dd6b0a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:12:59.870326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faac8b-d97c-424b-a049-f29a59dd6b0a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:12:59.870598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faac8b-d97c-424b-a049-f29a59dd6b0a None None] PATH_INFO: `/v3` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:12:59.874924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Mon Aug 8 00:12:59 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:12:59.875675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5572fc0a69c0 pid: 91987 (default app) Aug 08 00:13:08.396343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:08.396904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:08.396904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:08.398169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 2/2] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:08.405603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:08.405985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:08.406150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:08.477653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: INFO dbcounter [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] Registered counter for database keystone Aug 08 00:13:08.498861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG oslo_db.sqlalchemy.engines [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91987) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 00:13:08.504941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writer thread running {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 00:13:08.680669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG passlib.handlers.bcrypt [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91987) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 00:13:08.681099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG passlib.handlers.bcrypt [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91987) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 00:13:08.735669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:08.737258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:08.865829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1/3] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 00:13:08.988649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:09.068595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: INFO dbcounter [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] Registered counter for database keystone Aug 08 00:13:09.088293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG oslo_db.sqlalchemy.engines [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91988) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 00:13:09.094767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writer thread running {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 00:13:09.218012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef799fcd-c6aa-4b38-b6bc-6affb4758b77 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:09.222400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:09.223018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:09.223329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:09.223607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:09.232646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:09.285599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 00:13:09.285882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a0120-d8fd-4e37-8f59-5dc3d3826efd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:09.296541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 3/4] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Aug 8 00:13:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 00:13:09.302512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:09.317791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb29b60-aa95-46af-a123-a33682a9302d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:09.322225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:09.323353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:09.323635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:09.323905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] PATH_INFO: `/v3/projects/311e504f385e4c32bdb056e763a01299` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:09.325974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] RBAC: Authorizing `identity:get_project(project_id=311e504f385e4c32bdb056e763a01299)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:09.376058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING keystone.common.rbac_enforcer.enforcer [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 00:13:09.376395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ca5b-d929-4e22-bef1-7193b4f8cc39 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:09.380900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 2/5] 10.222.0.15 () {56 vars in 1207 bytes} [Mon Aug 8 00:13:09 2022] GET /identity/v3/projects/311e504f385e4c32bdb056e763a01299 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:11.382719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9668adc-eee7-429b-ba74-1915fe5f87ff None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:11.383308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9668adc-eee7-429b-ba74-1915fe5f87ff None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:11.383308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9668adc-eee7-429b-ba74-1915fe5f87ff None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:11.387547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 4/6] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:11.392918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:11.393364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:11.393458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:11.414446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:11.415989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:11.424544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 3/7] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 00:13:11.446736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9668adc-eee7-429b-ba74-1915fe5f87ff None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:11.459540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9668adc-eee7-429b-ba74-1915fe5f87ff None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:11.463725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a46233-6e1d-469c-8ee2-aa2e231c6c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:11.464315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46233-6e1d-469c-8ee2-aa2e231c6c53 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:11.464603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46233-6e1d-469c-8ee2-aa2e231c6c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:11.464885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46233-6e1d-469c-8ee2-aa2e231c6c53 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:11.472393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a46233-6e1d-469c-8ee2-aa2e231c6c53 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:11.474178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a46233-6e1d-469c-8ee2-aa2e231c6c53 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:11.483364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 5/8] 10.222.0.15 () {58 vars in 1324 bytes} [Mon Aug 8 00:13:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 00:13:11.489496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:11.498469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd7c2-d4b9-41a5-95ac-9ac1a83fb76f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:11.503332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895cdf8-168f-4d96-906b-966a4af8b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:11.503938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895cdf8-168f-4d96-906b-966a4af8b4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:11.504211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895cdf8-168f-4d96-906b-966a4af8b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:11.504477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895cdf8-168f-4d96-906b-966a4af8b4a1 admin admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:11.505770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895cdf8-168f-4d96-906b-966a4af8b4a1 admin admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:11.507590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895cdf8-168f-4d96-906b-966a4af8b4a1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:11.509080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 4/9] 10.222.0.15 () {56 vars in 1198 bytes} [Mon Aug 8 00:13:11 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:13.562680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6318-cb27-4a9c-b4b0-b4a4efbbd956 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.563215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6318-cb27-4a9c-b4b0-b4a4efbbd956 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.563215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6318-cb27-4a9c-b4b0-b4a4efbbd956 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.564847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 6/10] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:13.569954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.570363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.570505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.582878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821418-a604-4d2a-a4ef-8d0f80a9c93d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.583167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821418-a604-4d2a-a4ef-8d0f80a9c93d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.583487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb821418-a604-4d2a-a4ef-8d0f80a9c93d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.584780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 7/11] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:13.589015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:13.590553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.590830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.590926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:13.591102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.632751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG passlib.handlers.bcrypt [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91988) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 00:13:13.633187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG passlib.handlers.bcrypt [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91988) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 00:13:13.670731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 5/12] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:13.679333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:13.680930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:13.693567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:13.718642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:13.779848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 8/13] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:13.797090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958c3e7-5847-44cc-9543-5fb29dc46aa9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:13.801156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:13.831591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:13.832152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.832427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.832698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.843927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:13.852555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:13.853601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:13.854616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 6/14] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:13.860170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:13.904861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd09af-ffea-4742-9312-2d77448716b5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:13.907947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:13.941696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:13.944046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:13.944909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.945220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.945494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.959911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb79a84-f1a6-4521-95f2-20d742c03fd9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:13.994901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:13.995644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:13.996442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:13.996715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:13.997380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:13.997682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:13.999072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.010439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 7/15] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:14.013841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:14.015401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.016706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:14.119396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 9/16] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:14.125997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:14.129099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:14.221882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b69d-b2a6-4c86-ab65-b155529a7d7c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:14.224561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6cbe37-1109-41f0-a7f5-8604b3e9578c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:14.259593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:14.260587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:14.261053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:14.261797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:14.263573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:14.264254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:14.265541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:14.265990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.266079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:14.266441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:14.273013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 8/17] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:14 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:14.274006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:14.285456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:14.286927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.288238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:13:14.305114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:14.317888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:13:14.318698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:13:14.318698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:13:14.318698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:14.318698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:14.318698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:14.366755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:14.374664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f055449-5e77-41ea-b338-2347a62cfef9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:14.408626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:14.409255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:14.409532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:14.409805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:14.410915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:14.411014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0e394-0628-41d2-9901-79ead3110744 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:14.412419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.413957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:14.419671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:14.444619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:14.445246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:14.445525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:14.445800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:14.446438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:14.446743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:14.448175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56032d-7af6-4cf2-b119-7ec4ed6deea5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.466125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:14.502661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:14.523635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c48b9e-1521-4644-b1a4-8cd5053aa33a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:14.560451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:14.561070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:14.561347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:14.561620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:14.562356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.domain.id=default, user.id=de917b8f456944dc9d27423048f81375) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:14.562663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:14.564436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5d9bd-a20d-450c-bac1-feceb0741d99 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:14.572135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Aug 8 00:13:14 2022] GET /identity/v3/role_assignments?user.id=de917b8f456944dc9d27423048f81375&scope.domain.id=default&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:16.927572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a779-5d93-4ad4-9569-e8c17b00d899 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:16.928061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a779-5d93-4ad4-9569-e8c17b00d899 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:16.928061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6a779-5d93-4ad4-9569-e8c17b00d899 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:16.929553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:16.940550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181bb4-5642-49a3-a152-c841a2cb632d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:16.941564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181bb4-5642-49a3-a152-c841a2cb632d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:16.942084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181bb4-5642-49a3-a152-c841a2cb632d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:16.969152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c2181bb4-5642-49a3-a152-c841a2cb632d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:16.970707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c2181bb4-5642-49a3-a152-c841a2cb632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:17.047255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3538ec-54c2-4594-ad28-5638326563eb None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.047255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3538ec-54c2-4594-ad28-5638326563eb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.047255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3538ec-54c2-4594-ad28-5638326563eb None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.048166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 13/23] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:17.055056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.055626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.056102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.069487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 11/24] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:17.094910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29bc8-7bef-4177-9f7d-ee29d5d5f7ba None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.095191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29bc8-7bef-4177-9f7d-ee29d5d5f7ba None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.095671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29bc8-7bef-4177-9f7d-ee29d5d5f7ba None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.097137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 12/25] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:17.099429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:17.100654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:17.101936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.102431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.102908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.136332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:17.137908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:17.211145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 14/26] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:17.212013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:17.229677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:17.264016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:17.268980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:17.278518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:17.321446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95010a-700e-4222-9a68-9b4aa6d4d8f1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:17.365263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:17.366247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.366666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.367094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.379378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:17.381098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:13:17.396488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:17.397363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:13:17.398173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:13:17.398173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:13:17.398173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:17.399540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb98a0-4695-4c2f-b3ca-5b5db26fba3f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:17.400253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:17.401208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:17.437204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:17.437937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.438286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.438633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.439653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:17.441111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:17.617261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:17.627597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cbd9e-681e-4068-a34f-274f00133564 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:17.689294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88eede-e22a-4a3b-a4c9-87ed66796de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:17.690197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88eede-e22a-4a3b-a4c9-87ed66796de8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.690673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88eede-e22a-4a3b-a4c9-87ed66796de8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.691140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88eede-e22a-4a3b-a4c9-87ed66796de8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.694431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 14/29] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 00:13:17 2022] POST /identity/v3/roles => generated 219 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:13:17.700583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:17.721117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88eede-e22a-4a3b-a4c9-87ed66796de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-0d88eede-e22a-4a3b-a4c9-87ed66796de8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:17.729298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:17.730872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:17.731835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:17.735309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 16/30] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:17.736834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66dd40-0b9f-4198-bfd6-ea495a8b479f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.737326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66dd40-0b9f-4198-bfd6-ea495a8b479f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.737793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66dd40-0b9f-4198-bfd6-ea495a8b479f None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.739554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 17/31] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:17.745408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66dd40-0b9f-4198-bfd6-ea495a8b479f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:17.842989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:17.892256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397adc6-f1b2-4940-8671-3ae83998596a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:17.929379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66dd40-0b9f-4198-bfd6-ea495a8b479f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:17.931592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:17.932518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.932968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.934067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.935216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:17.935736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:17.937441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5266ca7-5cb1-4cfe-8ef0-4d3f67780469 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:17.961348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 15/32] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:17.963510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.963956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.964407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.984935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:17.986149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:17.986768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:17.990870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:17.990989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:17.991065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:17.991065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:17.991065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:17.991065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.003695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 18/33] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:18.007055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.026704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:18.080972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 16/34] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:18.082344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.129969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fbf95-9f5d-4d09-8b22-1ee98abe622a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:18.145933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:18.166690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:18.167353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.167724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.168005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.168895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:18.170418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.192575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daf03e-58c0-442b-a16e-c47bdf854a25 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:18.226350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e0c16-3195-4352-97bd-9efb7856916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:18.226977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e0c16-3195-4352-97bd-9efb7856916f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.227282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e0c16-3195-4352-97bd-9efb7856916f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.227556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e0c16-3195-4352-97bd-9efb7856916f None admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.228681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e0c16-3195-4352-97bd-9efb7856916f None admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:18.230301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e0c16-3195-4352-97bd-9efb7856916f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.231918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 17/35] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:18 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:18.236467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d10fa3-d9d2-451c-8a5c-a51f85b02a15 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.236811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d10fa3-d9d2-451c-8a5c-a51f85b02a15 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.237147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d10fa3-d9d2-451c-8a5c-a51f85b02a15 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.238392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 18/36] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:18.239216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d10fa3-d9d2-451c-8a5c-a51f85b02a15 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.355996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d10fa3-d9d2-451c-8a5c-a51f85b02a15 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:18.374224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 19/37] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:13:18 2022] POST /identity/v3/projects => generated 301 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:18.375442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.388774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:18.404342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1903c-f8db-4646-a796-7c1b88166f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:18.405043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1903c-f8db-4646-a796-7c1b88166f29 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.405687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1903c-f8db-4646-a796-7c1b88166f29 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.405947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1903c-f8db-4646-a796-7c1b88166f29 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.406737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1903c-f8db-4646-a796-7c1b88166f29 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:18.408488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1903c-f8db-4646-a796-7c1b88166f29 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.475968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:18.484548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 19/38] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 00:13:18 2022] POST /identity/v3/roles => generated 225 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:18.484548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ba86-7ed9-47aa-a8f9-1c0e9db39255 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.484548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ba86-7ed9-47aa-a8f9-1c0e9db39255 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.484548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ba86-7ed9-47aa-a8f9-1c0e9db39255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.506061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-dfa3ba86-7ed9-47aa-a8f9-1c0e9db39255 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:18.507925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-dfa3ba86-7ed9-47aa-a8f9-1c0e9db39255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:18.532807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d71e0-b26c-4548-b8af-9b160cab823e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:18.582077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53046db9-442b-4e93-abc0-2d3ea23ded03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:18.582743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53046db9-442b-4e93-abc0-2d3ea23ded03 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.584997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53046db9-442b-4e93-abc0-2d3ea23ded03 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.584997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53046db9-442b-4e93-abc0-2d3ea23ded03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.584997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53046db9-442b-4e93-abc0-2d3ea23ded03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:18.586715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53046db9-442b-4e93-abc0-2d3ea23ded03 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.588499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 20/39] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:18.590689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0fde0-b0b9-4104-bd53-2dbe4070805d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.591029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0fde0-b0b9-4104-bd53-2dbe4070805d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.591324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0fde0-b0b9-4104-bd53-2dbe4070805d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.592876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 21/40] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:18.597534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0fde0-b0b9-4104-bd53-2dbe4070805d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.616394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 20/41] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:18.616394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.616394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.616394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.639032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:18.640607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:18.720028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:18.720870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.739804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0fde0-b0b9-4104-bd53-2dbe4070805d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:18.768851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:18.776188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:18.781697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.781697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.781697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] PATH_INFO: `/v3/domains/default/users/de917b8f456944dc9d27423048f81375/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.781697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=de917b8f456944dc9d27423048f81375, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:18.781697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.824825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff92076-39c8-4411-b114-e0a4886f7c2c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:18.857403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:18.858040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:18.858316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:18.858588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:18.859257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:18.860649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:18.878702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:18.882607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1361 bytes} [Mon Aug 8 00:13:18 2022] PUT /identity/v3/domains/default/users/de917b8f456944dc9d27423048f81375/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:18.883462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:18.975367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 22/44] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 00:13:18 2022] POST /identity/v3/roles => generated 223 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:18.991181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8c7c7-53c5-41e7-81d0-8b04b4b98ebc None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:19.036768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0329ba-caad-4b60-b2cc-117dc5e5e3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:19.037725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0329ba-caad-4b60-b2cc-117dc5e5e3c6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:19.037725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0329ba-caad-4b60-b2cc-117dc5e5e3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:19.037908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0329ba-caad-4b60-b2cc-117dc5e5e3c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:19.038992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0329ba-caad-4b60-b2cc-117dc5e5e3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:19.040640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0329ba-caad-4b60-b2cc-117dc5e5e3c6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:19.042592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 23/45] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:19.049752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:19.160596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d8891-6c28-41e0-a149-fb67fce036c0 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:19.162165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:19.200159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a9ebf-d3ad-438d-bcac-baaf94bdee41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:19.200778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a9ebf-d3ad-438d-bcac-baaf94bdee41 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:19.201060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a9ebf-d3ad-438d-bcac-baaf94bdee41 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:19.201346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a9ebf-d3ad-438d-bcac-baaf94bdee41 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:19.201994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a9ebf-d3ad-438d-bcac-baaf94bdee41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:19.203533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a9ebf-d3ad-438d-bcac-baaf94bdee41 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:19.278299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:13:19 2022] POST /identity/v3/projects => generated 312 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:20.932091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345b47-c978-44eb-ba43-1ddeb2b884d5 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:20.932587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345b47-c978-44eb-ba43-1ddeb2b884d5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:20.932587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4345b47-c978-44eb-ba43-1ddeb2b884d5 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:20.934186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:20.939777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:20.940183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:20.940404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:20.959212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:20.960432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:21.024983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:21.047918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345b47-c978-44eb-ba43-1ddeb2b884d5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.115507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:21.149145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4345b47-c978-44eb-ba43-1ddeb2b884d5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.182714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2180a-ac57-4027-b978-69cb48ddaf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.183427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2180a-ac57-4027-b978-69cb48ddaf78 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.183721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2180a-ac57-4027-b978-69cb48ddaf78 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.183993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2180a-ac57-4027-b978-69cb48ddaf78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.194165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2180a-ac57-4027-b978-69cb48ddaf78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-2fc2180a-ac57-4027-b978-69cb48ddaf78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:21.199444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:21.205674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:21.206445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:21.207079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.268807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c6dc1-b566-49cd-920c-eec0d784a661 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.269089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c6dc1-b566-49cd-920c-eec0d784a661 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.269374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c6dc1-b566-49cd-920c-eec0d784a661 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.270707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 26/50] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:21.275710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.276005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.276277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.290022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:21.294420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:21.295994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:21.305694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34c1a9-b097-46b9-b064-a20f4bb50fcc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.341230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.341839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.342118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.342395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.343061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:21.343397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:21.344782 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:21.352718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 25/51] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:21.358250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.362887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 27/52] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:21.385145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.440908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:21.457441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ed62c-5df7-4d31-81a1-0376abe69bbd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.482845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24650d4d-788c-46f6-8cdf-faff82db4d74 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.491780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.492413 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.492690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.492963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.494105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:21.495591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:21.497422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:21 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:21.502593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.516633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.517264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.517545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.517818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.518185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:21.519007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:21.520473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:21.522035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 28/54] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:21.528306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.601591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afad8f-4d47-420c-9660-49ffb023918f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.626959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca36d59-17f9-4ab4-81ff-9e6ac3fbc00f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.635375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.636010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.636823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.637155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.638266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:21.639744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:21.641337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:21.646991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:21.654076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:21.663062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec581d0-0a95-43f5-b7e9-164bbdf143d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.663697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec581d0-0a95-43f5-b7e9-164bbdf143d1 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.663977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec581d0-0a95-43f5-b7e9-164bbdf143d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.664253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec581d0-0a95-43f5-b7e9-164bbdf143d1 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.664904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec581d0-0a95-43f5-b7e9-164bbdf143d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:21.666382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec581d0-0a95-43f5-b7e9-164bbdf143d1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:21.729456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:21.747539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4a2a6-78b4-4c3b-be6e-0fef65ad8245 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:21.778940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:21.779670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:21.779952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:21.780226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:21.780945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.domain.id=default, user.id=de917b8f456944dc9d27423048f81375) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:21.781251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:21.782684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479342d-f796-4681-8420-50dce2743ccf None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:21.788187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 28/56] 10.222.0.15 () {56 vars in 1322 bytes} [Mon Aug 8 00:13:21 2022] GET /identity/v3/role_assignments?user.id=de917b8f456944dc9d27423048f81375&scope.domain.id=default&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:21.795584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 29/57] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Aug 8 00:13:21 2022] POST /identity/v3/projects => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:13:23.861097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be6f6c-4ad8-49c4-a7f9-57f42f12e6b6 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:23.861613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be6f6c-4ad8-49c4-a7f9-57f42f12e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:23.861698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be6f6c-4ad8-49c4-a7f9-57f42f12e6b6 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:23.862991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 29/58] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:23.868171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:23.868535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:23.868736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:23.887047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:23.888352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:23.952046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 30/59] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:23.975661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be6f6c-4ad8-49c4-a7f9-57f42f12e6b6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:24.073875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be6f6c-4ad8-49c4-a7f9-57f42f12e6b6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:24.106331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7d712-0769-42e6-89c2-b730cde9749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:24.106941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7d712-0769-42e6-89c2-b730cde9749b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:24.107218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7d712-0769-42e6-89c2-b730cde9749b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:24.107519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7d712-0769-42e6-89c2-b730cde9749b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:24.108611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7d712-0769-42e6-89c2-b730cde9749b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:24.110056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7d712-0769-42e6-89c2-b730cde9749b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:24.111635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 30/60] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:24.117123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:24.214345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda994dd-6939-43cd-bb62-797d961d977e None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:24.223014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:24.248178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b2b80-76b5-4d56-be0d-9c8b0882e846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:24.248837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b2b80-76b5-4d56-be0d-9c8b0882e846 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:24.249136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b2b80-76b5-4d56-be0d-9c8b0882e846 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:24.249409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b2b80-76b5-4d56-be0d-9c8b0882e846 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:24.250117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b2b80-76b5-4d56-be0d-9c8b0882e846 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:24.251618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b2b80-76b5-4d56-be0d-9c8b0882e846 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:24.373230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 31/61] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:24 2022] POST /identity/v3/users => generated 275 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:27.552875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5d4c-725d-48ed-aa69-e879f9f8a882 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.553384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5d4c-725d-48ed-aa69-e879f9f8a882 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.553384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f5d4c-725d-48ed-aa69-e879f9f8a882 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.554740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 31/62] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:27.560797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.561216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.561406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.591639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:27.592870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:27.631549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52b8f8-9024-4000-8a3e-dc8bb91bef19 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.632163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52b8f8-9024-4000-8a3e-dc8bb91bef19 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.632647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52b8f8-9024-4000-8a3e-dc8bb91bef19 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.634295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 32/63] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:27.639936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf1c81-2f94-4b1b-919f-78534a4fa9c3 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.640489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf1c81-2f94-4b1b-919f-78534a4fa9c3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.640951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf1c81-2f94-4b1b-919f-78534a4fa9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.660327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-64cf1c81-2f94-4b1b-919f-78534a4fa9c3 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:27.661890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-64cf1c81-2f94-4b1b-919f-78534a4fa9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:27.699253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:27.702845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 32/64] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:27.740618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:27.769969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 33/65] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:27.771151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd4af7-5d90-40a1-a5d2-5140efff670a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.771475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd4af7-5d90-40a1-a5d2-5140efff670a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.771880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd4af7-5d90-40a1-a5d2-5140efff670a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.776979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 34/66] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:27.778628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.778907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.779180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.797510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:27.798737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:27.880288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 35/67] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:27.881179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:27.888793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:27.898690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7539050-0775-451b-98ae-41d0334a8a51 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:27.937670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb11c0d-e015-4f17-b081-0aa9325eebe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:27.938306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11c0d-e015-4f17-b081-0aa9325eebe9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.938586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11c0d-e015-4f17-b081-0aa9325eebe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.938859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11c0d-e015-4f17-b081-0aa9325eebe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.950454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb11c0d-e015-4f17-b081-0aa9325eebe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-8eb11c0d-e015-4f17-b081-0aa9325eebe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:27.955278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:27.956158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 33/68] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:27.960091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed4970-db4e-4adc-828f-49318e271a45 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:27.960785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed4970-db4e-4adc-828f-49318e271a45 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:27.960785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed4970-db4e-4adc-828f-49318e271a45 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:27.961298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 34/69] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:27.963004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed4970-db4e-4adc-828f-49318e271a45 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:27.972198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:27.997858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799e810d-72b1-4a2b-b896-2bbe0f0d2e6b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.032530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.033165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.033446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.033723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.044430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:28.049148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:28.049982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 36/70] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:13:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:28.055631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.078191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed4970-db4e-4adc-828f-49318e271a45 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.112757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4e44b-cb00-4a4a-86ce-6a5e2213ead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.113398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4e44b-cb00-4a4a-86ce-6a5e2213ead1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.113663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4e44b-cb00-4a4a-86ce-6a5e2213ead1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.113938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4e44b-cb00-4a4a-86ce-6a5e2213ead1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.124954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4e44b-cb00-4a4a-86ce-6a5e2213ead1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-97f4e44b-cb00-4a4a-86ce-6a5e2213ead1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:28.129669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:28.130552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:28.131751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 35/71] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:28.137260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.137540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.137812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.157100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:28.158443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:28.160556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d177b-1a52-4cff-bc03-7884572b143d None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.169196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.194793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.195443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.195724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.196001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.196668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:28.196972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.198367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.200498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.207935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 37/72] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.208869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.233498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 36/73] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:28.233498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.328539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad366c-866f-4a98-90fc-1b7a95db68f2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.336546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383136b-87cc-4c0c-ae22-0e4bc6006dc8 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.362177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.362781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.363060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.363365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.364022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:28.364323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.365696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.369847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.370501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.370782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.371062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.371768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:28.372073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.373431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.373520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 38/74] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.374275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.382176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 37/75] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.382915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.392636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.435882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.476361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0b249-f627-4cde-9cc3-ee1c26eed745 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.482559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c555bd1-4ae1-4b19-9a2c-6078ca268590 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.509731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9310a32c-8808-4768-8095-cf3e912de588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.510352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310a32c-8808-4768-8095-cf3e912de588 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.510638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310a32c-8808-4768-8095-cf3e912de588 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.510918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9310a32c-8808-4768-8095-cf3e912de588 None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.516312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.517076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.517353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.517623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.525309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310a32c-8808-4768-8095-cf3e912de588 None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.526826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9310a32c-8808-4768-8095-cf3e912de588 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.527645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:28.528684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 39/76] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.533393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6f441-d489-445c-9a9b-39c086a0bdf3 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.533393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6f441-d489-445c-9a9b-39c086a0bdf3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.533393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6f441-d489-445c-9a9b-39c086a0bdf3 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.533393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 40/77] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:28.533635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:28.534408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:28.535189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:28.535802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6f441-d489-445c-9a9b-39c086a0bdf3 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.535875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 38/78] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:28.535875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.606015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.634647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6f441-d489-445c-9a9b-39c086a0bdf3 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.639283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233555-f855-413b-b1b1-fa8574afe724 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.652325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.671177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.671808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.672096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.672372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.673485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.674838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a770034-43b7-4e35-8f01-d4c4bf03790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.674951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.675562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770034-43b7-4e35-8f01-d4c4bf03790a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.675902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770034-43b7-4e35-8f01-d4c4bf03790a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.676205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a770034-43b7-4e35-8f01-d4c4bf03790a None admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.676528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 41/79] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.677410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770034-43b7-4e35-8f01-d4c4bf03790a None admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.678926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a770034-43b7-4e35-8f01-d4c4bf03790a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.680664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 39/80] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.684303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.684782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.685074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.685349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.703208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:28.704407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:28.770477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 40/81] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:28.772480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.783875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0c4a7-d25a-4c83-a101-7318cbd9c51b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.817808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.818420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.818702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.818977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.844514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.873197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:28.881278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037f82d-065e-46fe-93fc-8bb176e46927 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:28.888989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.890399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.914690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:28.915326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:28.915612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:28.915884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:28.916545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:28.916846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:28.918223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:28.926087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 41/82] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:28.926087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:28.987739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 42/83] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:28.988590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.033481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f1984-eefa-4042-b7f4-3eb89169d1d5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.058511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.067391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44281936-ac86-434b-926f-ec091a93129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.068005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44281936-ac86-434b-926f-ec091a93129d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.068286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44281936-ac86-434b-926f-ec091a93129d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.068562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44281936-ac86-434b-926f-ec091a93129d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.074978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.085659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5e67f-6673-47a9-b966-d0a4494c5648 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.118713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.119351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.119632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.119908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.134292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44281936-ac86-434b-926f-ec091a93129d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.135813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44281936-ac86-434b-926f-ec091a93129d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.188869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.190513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.232870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 42/84] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:29.233708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44281936-ac86-434b-926f-ec091a93129d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.259638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.276598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.292164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 43/85] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:29.293668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.331189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44281936-ac86-434b-926f-ec091a93129d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.364089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.364700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.364981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.365253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.366356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.367831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.369551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 43/86] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:29.370138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.394993 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591aeea3-3318-4320-b11d-4b068eb464cc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.428876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.429499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.429779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.430055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.431181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.432664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.435214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 44/87] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:29.435214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.468240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.474584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62667cea-6d0c-4767-909e-1415b7dfd7d5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.493496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.508349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.508972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.509249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.509523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.510643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.512169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.513778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:29.518765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.542448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b100c-c4ab-4d89-8e89-c92e08478817 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.576535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.577169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.577457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.577731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.578861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.580364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.582007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 45/89] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:29.582741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.614280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac84e9-7b9e-4cc1-9555-e1c4b5a651a7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.648441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.649084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.649354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.649627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.650779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.652737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.654395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 45/90] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:29.655169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.686181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f4cec-d5f7-4e4f-a4df-5cbdf28b325f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.690266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.707073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.721940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.722480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.722758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.723027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.723724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.725158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.760615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975c4a0-3891-41cc-aae3-849b109ac4ac None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.796453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.797099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.797407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.797689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.811734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 46/91] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:29 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:29.812644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:29.891360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.892952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.925189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:29.926461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c53fb-59aa-44d6-ad91-42b0fe20c462 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:29.962881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:29.963525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:29.963804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:29.964076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:29.965371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=49a016ba549949ef81523de0e9a64871) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:29.965676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:29.967162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:29.973210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 47/92] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/role_assignments?user.id=49a016ba549949ef81523de0e9a64871&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:29.974043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:30.026540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:30.050587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 46/93] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:30.050878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:30.116128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d87ada-fac2-40b2-b6d9-583a073e5531 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:30.164428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:30.165053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:30.165334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:30.165782 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:30.166816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] RBAC: Adding query filter params (role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=49a016ba549949ef81523de0e9a64871) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:30.167128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:30.168734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:30.176786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:29 2022] GET /identity/v3/role_assignments?user.id=49a016ba549949ef81523de0e9a64871&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=1cef455d27914904ba2d1176ced4233c => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:30.177672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:30.190367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da51894-d73f-46f9-a400-ab5c271def96 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:30.235097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:30.241638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:30.242261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:30.242573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:30.242817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:30.243606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=de917b8f456944dc9d27423048f81375) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:30.250622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:30.250622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:30.255513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 47/95] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:30 2022] GET /identity/v3/role_assignments?user.id=de917b8f456944dc9d27423048f81375&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:30.316721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8fb876-4e45-4549-86e6-497052d98aaa None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:30.319846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:30.361578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256b5d5-cda2-4dab-a877-cc977ef18fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:30.362268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256b5d5-cda2-4dab-a877-cc977ef18fb1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:30.362551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256b5d5-cda2-4dab-a877-cc977ef18fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:30.362828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256b5d5-cda2-4dab-a877-cc977ef18fb1 None admin] PATH_INFO: `/v3/projects/0a8c45e418b948cd9877b80543764c4f` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:30.364161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256b5d5-cda2-4dab-a877-cc977ef18fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=0a8c45e418b948cd9877b80543764c4f)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:30.365745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256b5d5-cda2-4dab-a877-cc977ef18fb1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:30.367818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 49/96] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:30 2022] GET /identity/v3/projects/0a8c45e418b948cd9877b80543764c4f => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:30.374437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:30.494700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0512ac6-845b-45b5-a4de-ba918da31eb0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:30.534355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:30.534997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:30.535330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:30.535613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:30.536443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=0a8c45e418b948cd9877b80543764c4f, user.id=49a016ba549949ef81523de0e9a64871) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:30.536787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:30.538294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e8b7a-7b72-4817-98a6-96c2e8af9002 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:30.545635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 48/97] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:30 2022] GET /identity/v3/role_assignments?user.id=49a016ba549949ef81523de0e9a64871&scope.project.id=0a8c45e418b948cd9877b80543764c4f&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:32.657690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8635b-8ab2-4bf4-a9aa-b4e2f697ce26 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:32.658241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8635b-8ab2-4bf4-a9aa-b4e2f697ce26 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:32.658241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8635b-8ab2-4bf4-a9aa-b4e2f697ce26 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:32.659838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 50/98] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:32.665452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:32.666307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:32.666785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:32.689075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04788-d5f7-4227-a611-846ad614a7d4 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:32.689723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04788-d5f7-4227-a611-846ad614a7d4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:32.689885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a04788-d5f7-4227-a611-846ad614a7d4 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:32.691195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 51/99] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:32.696327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:32.696532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:32.696818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:32.697093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:32.697565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:32.752225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:32.753439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:32.770957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:32.797943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 49/100] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:32.835571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:32.926183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 52/101] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 00:13:32.954315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:32.984186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ede44-d6e5-4d3b-8caa-5bd3859be916 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.035410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.036059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.036339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.036690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.071070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:33.075886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:33.076763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:33.078374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 50/102] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:33.088576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.194059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:33.211677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa666961-2415-4f01-9aab-6b8f8dbbc93d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.213311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:33.268235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.268889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.269176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.269453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.289412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:33.295717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:33.298075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:33.300074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 53/103] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:13:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:33.304383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.339042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8efa7-0039-4f09-9b24-706f728574e1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.419440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.420058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.420336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.420605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.421304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:33.421604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:33.422990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a3caa-5c56-469e-b7d5-76300a1c15c7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 51/104] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fbece-f092-4374-ab84-08c574c584c1 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fbece-f092-4374-ab84-08c574c584c1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fbece-f092-4374-ab84-08c574c584c1 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 52/105] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e6c0-b4e9-4cd1-9911-c7376e3c518d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.475596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e6c0-b4e9-4cd1-9911-c7376e3c518d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.476216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4e6c0-b4e9-4cd1-9911-c7376e3c518d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.476216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 53/106] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:33.476216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e6c0-b4e9-4cd1-9911-c7376e3c518d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.485883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6d4a5-091e-4f63-aa72-d0095b5655bc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.523720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.524360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.524684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.524987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.525694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:33.526003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:33.528010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94a42b-33cf-4818-bb3c-13502c6f9344 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:33.540935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 54/107] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:33.540935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.540935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.540935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.557443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:33.558690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:33.571970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4e6c0-b4e9-4cd1-9911-c7376e3c518d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.577724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:33.582772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:33.609577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37894412-76bd-4915-942e-8d505cecf0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.610203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894412-76bd-4915-942e-8d505cecf0a2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.610477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894412-76bd-4915-942e-8d505cecf0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.610756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37894412-76bd-4915-942e-8d505cecf0a2 None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.611906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37894412-76bd-4915-942e-8d505cecf0a2 None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:33.613427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37894412-76bd-4915-942e-8d505cecf0a2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:33.615192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 54/108] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:33.620654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15972-e3a8-44aa-b53d-9b190d5dfe32 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.621012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15972-e3a8-44aa-b53d-9b190d5dfe32 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.621306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15972-e3a8-44aa-b53d-9b190d5dfe32 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.622954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 55/109] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:33.627996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.628289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.628569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.635540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 55/110] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:33.637103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.647581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:33.648806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:33.739346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 56/111] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:33.739346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.767265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1b527-4e8a-47da-84ba-37f10945c1ab None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.806301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10401d2-f0c6-41be-b0bb-905701fe9e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.806940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10401d2-f0c6-41be-b0bb-905701fe9e54 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.807219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10401d2-f0c6-41be-b0bb-905701fe9e54 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.807523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10401d2-f0c6-41be-b0bb-905701fe9e54 None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.809274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10401d2-f0c6-41be-b0bb-905701fe9e54 None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:33.810728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10401d2-f0c6-41be-b0bb-905701fe9e54 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:33.812336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 56/112] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:33.814725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.815002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.815297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.832227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:33.833420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:33.839171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbe9a1-7f98-4d88-b6bb-1f91ace85db0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:33.848591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:33.850122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:33.873771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:33.874426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:33.874698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:33.874978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:33.876199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:33.877732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:33.884600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 57/113] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:33.884600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.902192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 57/114] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:33.903053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:33.984056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce8183-9b6f-4e0e-b6a0-3312db97cee6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.001347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f7747-dcf7-446a-9f55-3995c1561a92 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.018275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41c73c-b3b0-481b-8ad9-5e03156967f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.018897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41c73c-b3b0-481b-8ad9-5e03156967f8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.019172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41c73c-b3b0-481b-8ad9-5e03156967f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.019471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41c73c-b3b0-481b-8ad9-5e03156967f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.029833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41c73c-b3b0-481b-8ad9-5e03156967f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:34.034313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ea41c73c-b3b0-481b-8ad9-5e03156967f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:34.034505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:34.035413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:34.036220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:34.036830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.036830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.036830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.037020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 58/115] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:34.041684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7463-f165-4621-9d9e-b7f1a597a6c0 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.042019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7463-f165-4621-9d9e-b7f1a597a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.042234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c7463-f165-4621-9d9e-b7f1a597a6c0 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.044782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 59/116] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:34.047345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7463-f165-4621-9d9e-b7f1a597a6c0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.051542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:34.056309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:34.057205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 58/117] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:34.058350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.073086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.084004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.148971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c7463-f165-4621-9d9e-b7f1a597a6c0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.159034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095d574-49fb-4dd0-aefe-6a86e71e470a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.183156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.183889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.184174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.184448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.185564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.187023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.188834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 60/118] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:34 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:34.189671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.195599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d10b49-16f4-4334-ae68-a06549a9d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.196263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d10b49-16f4-4334-ae68-a06549a9d38d None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.196547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d10b49-16f4-4334-ae68-a06549a9d38d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.196845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d10b49-16f4-4334-ae68-a06549a9d38d None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/49a016ba549949ef81523de0e9a64871/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.198861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d10b49-16f4-4334-ae68-a06549a9d38d None admin] RBAC: Authorizing `identity:create_grant(project_id=9804d64b9c1f43c59efe7e291a7b76e1, user_id=49a016ba549949ef81523de0e9a64871, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.200382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d10b49-16f4-4334-ae68-a06549a9d38d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.287696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb901b-8e44-4d12-b69f-673b35e9994a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.289934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.320456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.321075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.321355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.321630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.322701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.324284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.325821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 61/119] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:34.326582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.334722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 59/120] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:34 2022] PUT /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/49a016ba549949ef81523de0e9a64871/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:34.336087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.336367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.336675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.357262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:34.358644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:34.443263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 60/121] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:34.443263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.443263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.469428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3ee47-eb49-4327-8f1f-cb4b0c7a41c1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.512251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.512860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.513135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.513404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.514087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:34.514388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.516202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.525103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 62/122] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:34.532489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.561016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.561173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c82c8-ea62-4375-bde3-10ac3492e67d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.596314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.596924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.597207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.597477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.598146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:34.598488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.600417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.609543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 61/123] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:34.609543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.632929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73140676-a54e-44e7-bb07-8def4dedba8a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.672650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.673319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.673603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.673883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/49a016ba549949ef81523de0e9a64871/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.676087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] RBAC: Authorizing `identity:create_grant(project_id=9804d64b9c1f43c59efe7e291a7b76e1, user_id=49a016ba549949ef81523de0e9a64871, role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.677554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.683635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.722466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02132767-2ca7-458c-93d4-319801cf6a1f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.755160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.755897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.756226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.756501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.757142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:34.758586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:34.780435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 63/124] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:34 2022] PUT /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/49a016ba549949ef81523de0e9a64871/roles/1cef455d27914904ba2d1176ced4233c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:34.784099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.887371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 62/125] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:13:34 2022] POST /identity/v3/projects => generated 302 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:34.891324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:34.902979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:34.921366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdad1f-98e3-4549-894a-1fb3164bb12d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:34.978386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:34.979094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:34.979502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:34.980574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:34.995250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:35.001593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:35.002529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:35.003403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:35.005740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:35.006833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.076873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79385b70-f6be-4cf7-8ea1-ba614c0f841e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.097200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:35.118203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.118873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.119248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.119602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.121532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.123120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.125091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 63/127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:34 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:35.125804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.135037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8e96b-85f4-446e-b58a-1dd1e197d142 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.174319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.174932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.175256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.175533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.176685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.178150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.179978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 65/128] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:35 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:35.180579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.207058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:35.246113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d46a-c7fb-4132-ab92-7f54638b1dd3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.288723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ea79f-619d-4fa5-9797-e7824097f247 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.293808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.294628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.295073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.295508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.296330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] RBAC: Adding query filter params (name=reader) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:35.296785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.298521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.318656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 64/129] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:35.319857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.328760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.329527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.329884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.330265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] PATH_INFO: `/v3/projects/0a8c45e418b948cd9877b80543764c4f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.331573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] RBAC: Authorizing `identity:get_project(project_id=0a8c45e418b948cd9877b80543764c4f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.333212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.335494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 66/130] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:35 2022] GET /identity/v3/projects/0a8c45e418b948cd9877b80543764c4f => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:35.338764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.414381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:35.433553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:35.444090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d5d52-4a7b-4ae6-aba2-048aa0a1b168 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.482605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.483377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.483741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.485181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] PATH_INFO: `/v3/users/f46eec5ccc554e078093d4892442748e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.515936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] RBAC: Authorizing `identity:get_user(user_id=f46eec5ccc554e078093d4892442748e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.518217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.520189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 67/131] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:35 2022] GET /identity/v3/users/f46eec5ccc554e078093d4892442748e => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:35.522199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.532841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b4fa5-3b4c-49c6-a6b9-900b8f54621e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.608924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.609674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.609853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.610115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.611290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.612758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.614625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 65/132] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:35 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:35.619128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.669559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde128e9-7dcc-4a77-9cf0-d96d8df0933f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.739627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.740765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.741051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.741329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] PATH_INFO: `/v3/projects/0a8c45e418b948cd9877b80543764c4f/users/49a016ba549949ef81523de0e9a64871/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.744395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a8c45e418b948cd9877b80543764c4f, user_id=49a016ba549949ef81523de0e9a64871, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.745915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.784491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575483-ab37-4ddd-a292-afd93a8772aa None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:35.801357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:35.822260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:35.822872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:35.823194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:35.823636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:35.880452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 68/133] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:35 2022] PUT /identity/v3/projects/0a8c45e418b948cd9877b80543764c4f/users/49a016ba549949ef81523de0e9a64871/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:35.882684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:35.895130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:35.896678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:35.907675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:36.026129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 66/134] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:36.028332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:36.050745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:36.144448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350f521-2c01-4c70-b651-9d293e582354 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:36.161675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06188-816c-4138-b216-bb9b59b91205 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:36.180153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:36.180784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:36.181060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:36.181335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:36.182471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:36.184017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:36.190779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 67/135] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:36 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:36.192908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:36.229462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98ddb4-3539-47b2-9d0b-78e0668a66dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:36.230283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ddb4-3539-47b2-9d0b-78e0668a66dd None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:36.230619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ddb4-3539-47b2-9d0b-78e0668a66dd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:36.230900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ddb4-3539-47b2-9d0b-78e0668a66dd None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/de917b8f456944dc9d27423048f81375/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:36.232944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98ddb4-3539-47b2-9d0b-78e0668a66dd None admin] RBAC: Authorizing `identity:create_grant(project_id=9804d64b9c1f43c59efe7e291a7b76e1, user_id=de917b8f456944dc9d27423048f81375, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:36.234405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98ddb4-3539-47b2-9d0b-78e0668a66dd None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:36.280896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:36.295687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275a3db-1b5f-46a2-b9ce-337c141b7388 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:36.332435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:36.333048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:36.333326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:36.333653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:36.334390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] RBAC: Adding query filter params (role.id=248e2f3c1da941b8bc7e3551815d06a7, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=f46eec5ccc554e078093d4892442748e) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:36.334695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:36.336244 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e535f-7d35-4b4f-801e-402e12802f42 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:36.337969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 69/136] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:35 2022] PUT /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/de917b8f456944dc9d27423048f81375/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:36.342406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 68/137] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:36 2022] GET /identity/v3/role_assignments?user.id=f46eec5ccc554e078093d4892442748e&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=248e2f3c1da941b8bc7e3551815d06a7 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:36.926896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8ff72-6d0f-4bf8-af46-7da4e732f1e6 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:36.927431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8ff72-6d0f-4bf8-af46-7da4e732f1e6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:36.927431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8ff72-6d0f-4bf8-af46-7da4e732f1e6 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:36.928752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 70/138] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:36.934699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:36.935159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:36.935410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:36.964913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:36.966154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:37.038445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 69/139] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:37.061852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8ff72-6d0f-4bf8-af46-7da4e732f1e6 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:37.147763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:37.190925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8ff72-6d0f-4bf8-af46-7da4e732f1e6 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:37.236878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:37.237586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:37.237867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:37.238139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:37.258067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:37.263515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:37.264363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:37.265144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 71/140] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:37.271883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:37.368035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:37.392025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509ec6d-ca64-4136-89c8-49b83337ac85 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:37.427312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:37.427923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:37.428199 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:37.428473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:37.429132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:37.429429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:37.430810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:37.439637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 70/141] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:37.445494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:37.563283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9d023-9e72-4b4d-a996-7f78ed76387c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:37.601499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:37.609958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b543d-1fc9-4cfd-a9c0-eac3f58f764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:37.610597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b543d-1fc9-4cfd-a9c0-eac3f58f764c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:37.613060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b543d-1fc9-4cfd-a9c0-eac3f58f764c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:37.613060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b543d-1fc9-4cfd-a9c0-eac3f58f764c None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:37.613060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b543d-1fc9-4cfd-a9c0-eac3f58f764c None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:37.613734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b543d-1fc9-4cfd-a9c0-eac3f58f764c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:37.615339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 72/142] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:37 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:37.621023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:37.673418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647a842-d700-4764-9a3f-51993fba996c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:37.673906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647a842-d700-4764-9a3f-51993fba996c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:37.674358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647a842-d700-4764-9a3f-51993fba996c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:37.676206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 73/143] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:37.681236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:37.681719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:37.682341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:37.709028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:37.711157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:37.775171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e614dc8-d9d4-4a9c-9ea9-f63450f27a2d None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:37.847805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e439d0-200b-488a-b554-aeb706c6784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:37.848744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e439d0-200b-488a-b554-aeb706c6784c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:37.849173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 74/144] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:37.849655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e439d0-200b-488a-b554-aeb706c6784c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:37.850121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e439d0-200b-488a-b554-aeb706c6784c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:37.862824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:37.912332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:37.963724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e439d0-200b-488a-b554-aeb706c6784c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:37.965683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e439d0-200b-488a-b554-aeb706c6784c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:38.075014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16ee73-dd8f-4fc9-8f70-d773eed1b83d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:38.101369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 71/145] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:38.107126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70efd0-0051-4fa6-b6fd-cfbbfc016ee8 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.107434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70efd0-0051-4fa6-b6fd-cfbbfc016ee8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.107704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70efd0-0051-4fa6-b6fd-cfbbfc016ee8 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.108967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 72/146] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:38.110644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70efd0-0051-4fa6-b6fd-cfbbfc016ee8 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.124132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:38.134252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9281d92-d3b2-410c-bbe5-0a54a3b8352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:38.135509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9281d92-d3b2-410c-bbe5-0a54a3b8352b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.136499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9281d92-d3b2-410c-bbe5-0a54a3b8352b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.139424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9281d92-d3b2-410c-bbe5-0a54a3b8352b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.139424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9281d92-d3b2-410c-bbe5-0a54a3b8352b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:38.140413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9281d92-d3b2-410c-bbe5-0a54a3b8352b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:38.142179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 75/147] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:38.143649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.144129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.144574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.166673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:38.168287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:38.190173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:38.259340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70efd0-0051-4fa6-b6fd-cfbbfc016ee8 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:38.298891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:38.299901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.300372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.300852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.302243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:38.304002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:38.305823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 73/148] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:38 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:38.306811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.320663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 76/149] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:38.321686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.417872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd50bb5d-0271-4246-9e8e-b883a1229bea None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:38.431869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:38.475332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5ba0f-0ee9-4977-8849-6ee3b9e6e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:38.475942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ba0f-0ee9-4977-8849-6ee3b9e6e920 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.476217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ba0f-0ee9-4977-8849-6ee3b9e6e920 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.476484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5ba0f-0ee9-4977-8849-6ee3b9e6e920 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.477133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5ba0f-0ee9-4977-8849-6ee3b9e6e920 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:38.481856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5ba0f-0ee9-4977-8849-6ee3b9e6e920 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:38.507084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb66de-c8f4-47e0-863e-6efa6cf7e0a2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:38.542381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:38.542922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.543421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.543709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.544424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=49a016ba549949ef81523de0e9a64871) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:38.544729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:38.546222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:38.552514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:38.556009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 77/150] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:38 2022] GET /identity/v3/role_assignments?user.id=49a016ba549949ef81523de0e9a64871&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:38.556868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.623627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 74/151] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:38 2022] POST /identity/v3/users => generated 283 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:38.698318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f404ed-e4bd-4556-96d8-dcdfa97e8634 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:38.710076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c264a9-e50d-4b15-9182-0a5abca7fea6 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.710710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c264a9-e50d-4b15-9182-0a5abca7fea6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.711179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c264a9-e50d-4b15-9182-0a5abca7fea6 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.712960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 75/152] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:38.718967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.719603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.720073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.748921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:38.752817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:38.761740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:38.763073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.765873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.766428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.792669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:38.798953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:38.799884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:38.800642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 78/153] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:13:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:38.806465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.849651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 76/154] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:38.875352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:38.877131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.915574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64cd15c-9c1a-4d18-9ff4-049604f8d310 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:38.963561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:38.964847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:38.965131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:38.965406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:38.966081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:38.966382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:38.968800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:38.977271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 79/155] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:13:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:38.982903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:38.990377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.003852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e06d18-ea34-437d-8d36-33e2a8da78a0 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.047695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.048263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.048563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.049299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.061600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:39.067989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:39.068872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:39.069701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 77/156] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:39.081226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:39.104501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a54ba7-62b1-4a62-908a-2ef72565d5a6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.126879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.142120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.143041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.143572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.144067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.145412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:39.147045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:39.148911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 80/157] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:38 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:39.157190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:39.208044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e703da-382b-4fd9-b157-f310d4ccbd69 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.260517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.261828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.262367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.262866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.263947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:39.264755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:39.266591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:39.282671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 78/158] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:39.290591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:39.297130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9dd5ec-5df4-4af4-85f2-a61db603326a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.315889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.349247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a4e0b-d626-44e4-94fe-6d6d32d3130c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.350845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a4e0b-d626-44e4-94fe-6d6d32d3130c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.355433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a4e0b-d626-44e4-94fe-6d6d32d3130c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.356025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a4e0b-d626-44e4-94fe-6d6d32d3130c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.427662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.446020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e366b-b1df-4bc1-a5e4-3b8a6f9e7dbe None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.466981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a4e0b-d626-44e4-94fe-6d6d32d3130c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:39.468992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a4e0b-d626-44e4-94fe-6d6d32d3130c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:39.498791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480112b-791c-4922-ae94-cb68576cdc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.499548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480112b-791c-4922-ae94-cb68576cdc21 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.499827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480112b-791c-4922-ae94-cb68576cdc21 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.500096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480112b-791c-4922-ae94-cb68576cdc21 None admin] PATH_INFO: `/v3/users/49a016ba549949ef81523de0e9a64871` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.501276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480112b-791c-4922-ae94-cb68576cdc21 None admin] RBAC: Authorizing `identity:get_user(user_id=49a016ba549949ef81523de0e9a64871)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:39.502850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480112b-791c-4922-ae94-cb68576cdc21 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:39.504538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 79/159] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:39 2022] GET /identity/v3/users/49a016ba549949ef81523de0e9a64871 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:39.505818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82747e-9afa-4c23-b1ac-f36abf41cdd7 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.506099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82747e-9afa-4c23-b1ac-f36abf41cdd7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.506369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82747e-9afa-4c23-b1ac-f36abf41cdd7 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.507597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 80/160] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:39.512550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82747e-9afa-4c23-b1ac-f36abf41cdd7 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:39.596590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 81/161] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:39.597907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.598193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.603102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.623256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:39.624597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:39.629551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.673890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82747e-9afa-4c23-b1ac-f36abf41cdd7 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.722507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 82/162] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:39.723887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:39.729753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.730840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.731242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.731561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.740627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.860087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:39.861511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:39.892359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b707e0-2053-444c-9d0f-11bc95ca78e5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:39.957115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de9600-dbef-43ff-b30e-89ea28181874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:39.957769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de9600-dbef-43ff-b30e-89ea28181874 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.958045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de9600-dbef-43ff-b30e-89ea28181874 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.958311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de9600-dbef-43ff-b30e-89ea28181874 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.959545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de9600-dbef-43ff-b30e-89ea28181874 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:39.960946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de9600-dbef-43ff-b30e-89ea28181874 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:39.962998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 83/163] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:39 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:39.964353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3cc8b-bafd-4302-8ef1-503b48682222 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:39.964856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3cc8b-bafd-4302-8ef1-503b48682222 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:39.965141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3cc8b-bafd-4302-8ef1-503b48682222 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:39.972050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:39.973151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 84/164] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:39.973960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3cc8b-bafd-4302-8ef1-503b48682222 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.005195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 81/165] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:40.006051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.136969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ea619-ea4b-49cb-bf25-5600848a360b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.159953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:40.193675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.194345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.194622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.194923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.195687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] RBAC: Adding query filter params (role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=49a016ba549949ef81523de0e9a64871) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:40.196027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:40.197439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1a809-5d50-41e7-a767-6e99b1af7c26 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:40.204560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3cc8b-bafd-4302-8ef1-503b48682222 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.209666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 82/166] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/role_assignments?user.id=49a016ba549949ef81523de0e9a64871&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=1cef455d27914904ba2d1176ced4233c => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:40.214061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.214348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.214633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.251448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:40.252659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:40.271008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.271729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.271997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.272259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.291367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:40.296107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:40.296988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:40.297969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 85/167] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:40.302656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.320771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:40.363909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 83/168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:40.365554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.460249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba23eb-b056-440c-8cad-31db454f4510 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.509407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754e16f-e784-405b-945e-0dd67ebc2e1e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.516985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.517887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.518307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.518743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] PATH_INFO: `/v3/projects/0a8c45e418b948cd9877b80543764c4f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.520176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] RBAC: Authorizing `identity:get_project(project_id=0a8c45e418b948cd9877b80543764c4f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:40.521814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:40.523623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 86/169] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/projects/0a8c45e418b948cd9877b80543764c4f => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:40.524525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.532661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:40.551041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.551966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.552555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.552932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.553708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:40.554115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:40.555623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:40.565740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 84/170] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:40.566428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.610182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:40.687590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c99a6-f33a-46ed-909f-05306b586c2f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.724793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.725339 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.725620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.725891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.726620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=0a8c45e418b948cd9877b80543764c4f, user.id=49a016ba549949ef81523de0e9a64871) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:40.726927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:40.728440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:40.735735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 85/171] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/role_assignments?user.id=49a016ba549949ef81523de0e9a64871&scope.project.id=0a8c45e418b948cd9877b80543764c4f&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:40.736567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.810029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5fcfc-323f-4f54-ad80-4aa0c763b8ca None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.850108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd4137-19d8-466b-b93f-faf468dbbcc4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:40.853234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:40.890996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.891688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.891970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.892248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] PATH_INFO: `/v3/users/f46eec5ccc554e078093d4892442748e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.893426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] RBAC: Authorizing `identity:get_user(user_id=f46eec5ccc554e078093d4892442748e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:40.894959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:40.896828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 86/172] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/users/f46eec5ccc554e078093d4892442748e => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:40.906472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:40.920619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:40.921271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:40.921549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:40.921819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:40.948788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:40.962128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:40.963083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:40.964614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 87/173] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:40.984378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:41.031724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a18e90-0bf0-4eaa-a6cf-7bf398fe4376 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:41.083649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:41.118819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:41.119496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.119778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.120051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.121635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be00ca-f802-4448-90e9-abdd62d51360 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:41.122020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:41.123549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:41.125575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 87/174] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:41.140729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:41.177792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:41.178423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.178663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.178929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.179616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:41.179912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:41.181607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:41.200262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 88/175] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:41.207199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:41.242424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:41.366272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f76a6e-dbfe-45fa-8ef9-a43c5d04debc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:41.372918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded0696-19de-474f-a0df-4c63262db48f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:41.423608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:41.435895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:41.436572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.436849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.437117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/f46eec5ccc554e078093d4892442748e/roles/248e2f3c1da941b8bc7e3551815d06a7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.440207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] RBAC: Authorizing `identity:create_grant(project_id=9804d64b9c1f43c59efe7e291a7b76e1, user_id=f46eec5ccc554e078093d4892442748e, role_id=248e2f3c1da941b8bc7e3551815d06a7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:41.441655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:41.445693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:41.446328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.446599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.446865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] PATH_INFO: `/v3/users/de917b8f456944dc9d27423048f81375` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.448755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] RBAC: Authorizing `identity:get_user(user_id=de917b8f456944dc9d27423048f81375)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:41.450203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:41.451836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 89/176] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:41 2022] GET /identity/v3/users/de917b8f456944dc9d27423048f81375 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:41.457625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:41.546650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 88/177] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:41 2022] PUT /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/users/f46eec5ccc554e078093d4892442748e/roles/248e2f3c1da941b8bc7e3551815d06a7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:41.579638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac83868-57e5-4f3f-8043-8ec21dffb387 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:41.618669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeefdb-c183-495e-8478-db59677f8c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:41.619768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeefdb-c183-495e-8478-db59677f8c39 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.620850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeefdb-c183-495e-8478-db59677f8c39 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.621329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeefdb-c183-495e-8478-db59677f8c39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.707716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:41.718705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeefdb-c183-495e-8478-db59677f8c39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:41.723691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeefdb-c183-495e-8478-db59677f8c39 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:41.830675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 90/178] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:41.840418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:41.914319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c6af5-6ebb-4209-99bf-3da2a392c4ce None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.914625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c6af5-6ebb-4209-99bf-3da2a392c4ce None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.915148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c6af5-6ebb-4209-99bf-3da2a392c4ce None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.916605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 91/179] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:41.923385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:41.923719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:41.924003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:41.957649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:41.958851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:42.001402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.006277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d01b60b-40d8-48a2-ba6d-60b62c62385e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.034699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 92/180] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:42.048937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.050246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.050801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.051305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.052819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:42.054618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:42.057208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 89/181] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:41 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:42.061628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.066069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.087954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.225302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52499654-77ce-4d8c-86c4-e80e721056ee None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.243529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811db0c-d6c8-418b-8334-e90c5ec1ed99 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.270447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e09495-597a-4edd-9333-4db7fb6f1ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.270447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09495-597a-4edd-9333-4db7fb6f1ba3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.270447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09495-597a-4edd-9333-4db7fb6f1ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.270447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e09495-597a-4edd-9333-4db7fb6f1ba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.284127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e09495-597a-4edd-9333-4db7fb6f1ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-12e09495-597a-4edd-9333-4db7fb6f1ba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:42.289043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:42.289936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:42.290725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 93/182] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:42.291434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.291520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30613-145e-4f6c-98f7-6cef5726f191 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.292125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30613-145e-4f6c-98f7-6cef5726f191 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.292125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f30613-145e-4f6c-98f7-6cef5726f191 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.297793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 94/183] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:42.298327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.298716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.299206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.299343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f30613-145e-4f6c-98f7-6cef5726f191 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.300297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=de917b8f456944dc9d27423048f81375) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:42.300613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:42.302149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c019c-98e6-48b3-a30d-5729d5f3492b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:42.310890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 90/184] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/role_assignments?user.id=de917b8f456944dc9d27423048f81375&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:42.312728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f8683-a0f3-4703-b4ba-a3b326fe65c6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.313398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f8683-a0f3-4703-b4ba-a3b326fe65c6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.313691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0f8683-a0f3-4703-b4ba-a3b326fe65c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.341086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ba0f8683-a0f3-4703-b4ba-a3b326fe65c6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:42.343515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ba0f8683-a0f3-4703-b4ba-a3b326fe65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:42.375974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.406605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.427401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f30613-145e-4f6c-98f7-6cef5726f191 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.443349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 91/185] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:42.455482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6f000-be23-41ea-8ddb-4b1ae97756bd None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.455482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6f000-be23-41ea-8ddb-4b1ae97756bd None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.455482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6f000-be23-41ea-8ddb-4b1ae97756bd None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.455482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 92/186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:42.460866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.461154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.461597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.487613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.488612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.488870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:42.489319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.489832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.490088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:42.490896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:42.491198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:42.492625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:42.503546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 95/187] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:42.504391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.583204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 93/188] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:42.584100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.631185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a8377-925f-4181-a99d-9cbecb2e2c64 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.673299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.673949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.674230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.674499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.685276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:42.690328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:42.690719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:42.691594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:42.692383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 96/189] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:42.693022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.702255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.707413 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7febf7-5fa4-4a67-95b9-377650d3ef4f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.741259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.741881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.742250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.742529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] PATH_INFO: `/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.755895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] RBAC: Authorizing `identity:get_user(user_id=a0e598ea6e5d4f258ff25040a0fec9f4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:42.757359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:42.764284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 94/190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:42.764284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.798688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c61a5-d5d4-40ad-bc93-05dbb85735db None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.831611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.832232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.832507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.832773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.833853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:42.835303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:42.841549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 97/191] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:42.841549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.877786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2909-10f2-4dc1-9e3a-f097f65cd96e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.912310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.914176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.914835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.915112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.915399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:42.916110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:42.916407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:42.918471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:42.926733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 95/192] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:42.927420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:42.940968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:42.944261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e118e8a-ca18-4625-9c5f-8bc767f96c6f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:42.981526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:42.982130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:42.982408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:42.982677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.037489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3381d9-a742-4a79-9b56-aab1a6c7c903 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:43.052798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.054628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.074230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:43.075019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:43.075357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:43.075633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.076272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.077821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.126703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:43.151338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 98/193] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:43.152730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:43.203313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 96/194] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:42 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:43.204049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:43.289597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:43.293291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf6bf74-3f8b-4cdd-86be-14e511b64b8a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:43.334274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8ea19-cb21-4cad-a642-df9084d08dcb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:43.341888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:43.343584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:43.344078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:43.344889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] PATH_INFO: `/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.346707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] RBAC: Authorizing `identity:get_user(user_id=a0e598ea6e5d4f258ff25040a0fec9f4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.349155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.352308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 99/195] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:43 2022] GET /identity/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:43.363287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:43.380144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93858036-78b1-4952-b828-2045b2944308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:43.381134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93858036-78b1-4952-b828-2045b2944308 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:43.381551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93858036-78b1-4952-b828-2045b2944308 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:43.382004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93858036-78b1-4952-b828-2045b2944308 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.383321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93858036-78b1-4952-b828-2045b2944308 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.384836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93858036-78b1-4952-b828-2045b2944308 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.387862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 97/196] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:43 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:43.393283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93858036-78b1-4952-b828-2045b2944308 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:43.416314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:43.483452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dac2ab-cfa4-4399-b20e-5930c58f5905 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:43.515684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93858036-78b1-4952-b828-2045b2944308 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:43.525164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:43.525774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:43.526047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:43.526312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.574203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:43.575113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:43.575520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:43.575871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.577351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=a0e598ea6e5d4f258ff25040a0fec9f4) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:43.577351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.579007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.581127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:43.585822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 98/197] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:43 2022] GET /identity/v3/role_assignments?user.id=a0e598ea6e5d4f258ff25040a0fec9f4&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:43.621077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.623191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.687747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:43.744950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 100/198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:43.751164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:43.861401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba9e02-770c-461a-93b2-ae0de1db13fb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:43.895710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e33631-ef5f-4ad1-b127-904e441c79f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:43.896353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e33631-ef5f-4ad1-b127-904e441c79f4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:43.896631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e33631-ef5f-4ad1-b127-904e441c79f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:43.896905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e33631-ef5f-4ad1-b127-904e441c79f4 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:43.898050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e33631-ef5f-4ad1-b127-904e441c79f4 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:43.899530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e33631-ef5f-4ad1-b127-904e441c79f4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:43.902022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 99/199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:43 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:43.907970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:44.058464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a855ae-669a-4965-8c8f-9c2bf88b0281 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:44.102807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:44.103447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:44.103724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:44.103992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:44.105785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] RBAC: Adding query filter params (role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=a0e598ea6e5d4f258ff25040a0fec9f4) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:44.106163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:44.107622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333974e-d5c7-4b92-b81f-c0634bfa1a76 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:44.118964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 101/200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:43 2022] GET /identity/v3/role_assignments?user.id=a0e598ea6e5d4f258ff25040a0fec9f4&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=1cef455d27914904ba2d1176ced4233c => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:44.193863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d047-7da0-4c54-a405-029046617dc9 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:44.194907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d047-7da0-4c54-a405-029046617dc9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:44.195452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa7d047-7da0-4c54-a405-029046617dc9 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:44.197113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 100/201] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:44.203539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:44.204160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:44.204649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:44.219416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:44.225567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:44.227138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:44.313202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 102/202] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:44.358581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7d047-7da0-4c54-a405-029046617dc9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:44.446822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:44.492206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa7d047-7da0-4c54-a405-029046617dc9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:44.540957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:44.542056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:44.542565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:44.543037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:44.554705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:44.560885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:44.561842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:44.562627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:44.570304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 101/203] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:44.571922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:44.707252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea3191-3f24-4ffe-8c00-435aa71d2f7d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:44.760897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:44.761725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:44.762004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:44.762275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:44.762952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:44.763267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:44.764654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:44.771763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:44.779498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 103/204] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:44.785374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:44.955257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be908152-0265-4c7d-b203-fd997a006730 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:45.006572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:45.034470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:45.035764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:45.036280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:45.036643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] PATH_INFO: `/v3/users/f46eec5ccc554e078093d4892442748e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:45.045093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] RBAC: Authorizing `identity:get_user(user_id=f46eec5ccc554e078093d4892442748e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:45.045550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:45.049084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 102/205] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:44 2022] GET /identity/v3/users/f46eec5ccc554e078093d4892442748e => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:45.063275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:45.168316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5074ea-5b3a-4f1d-a66e-5404faf98e84 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:45.210753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:45.211313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:45.211590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:45.215249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:45.312493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:45.322813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:45.324958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:45.459256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 104/206] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:45.471329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:45.647977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982d908-2e9b-4e1b-ae11-bc8ab912e822 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:45.707641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63219e0-4166-4df4-84e0-a23d499c0868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:45.708308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63219e0-4166-4df4-84e0-a23d499c0868 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:45.708593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63219e0-4166-4df4-84e0-a23d499c0868 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:45.708871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63219e0-4166-4df4-84e0-a23d499c0868 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:45.710032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63219e0-4166-4df4-84e0-a23d499c0868 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:45.711580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63219e0-4166-4df4-84e0-a23d499c0868 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:45.713614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 103/207] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:45 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:45.720200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:45.860191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68ed8e-9d1a-4a9d-bedb-dcf361127849 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:45.864783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:45.906340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:45.906961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:45.907261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:45.907528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:45.908260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] RBAC: Adding query filter params (role.id=248e2f3c1da941b8bc7e3551815d06a7, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1, user.id=f46eec5ccc554e078093d4892442748e) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:45.911592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:45.911592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cd856-b6b2-4598-8697-7171d4a760d2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:45.922674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 105/208] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:45 2022] GET /identity/v3/role_assignments?user.id=f46eec5ccc554e078093d4892442748e&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=248e2f3c1da941b8bc7e3551815d06a7 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:45.992149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106ad8-e407-4fdd-9617-5f691e3887ec None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:45.992464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106ad8-e407-4fdd-9617-5f691e3887ec None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:45.992715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f106ad8-e407-4fdd-9617-5f691e3887ec None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:45.994220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 104/209] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:46.005272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c651-3210-4dda-b696-a6039290f040 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.005575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c651-3210-4dda-b696-a6039290f040 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.005824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8c651-3210-4dda-b696-a6039290f040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.029840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6bf8c651-3210-4dda-b696-a6039290f040 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:46.031081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6bf8c651-3210-4dda-b696-a6039290f040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:46.125211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 106/210] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:46.149241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f106ad8-e407-4fdd-9617-5f691e3887ec None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.222125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:46.265986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f106ad8-e407-4fdd-9617-5f691e3887ec None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.301457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ca93d-ccde-49d1-b994-464f49753268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.302093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ca93d-ccde-49d1-b994-464f49753268 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.302373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ca93d-ccde-49d1-b994-464f49753268 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.302685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ca93d-ccde-49d1-b994-464f49753268 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.314057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ca93d-ccde-49d1-b994-464f49753268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:46.317578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8236-321d-4c82-890a-bc5b5d1091f5 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.317865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8236-321d-4c82-890a-bc5b5d1091f5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.318135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf8236-321d-4c82-890a-bc5b5d1091f5 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b15ca93d-ccde-49d1-b994-464f49753268 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:46.319165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:46.321161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:46.323115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:46.325552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 107/211] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:46.326014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 105/212] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:46.330015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf8236-321d-4c82-890a-bc5b5d1091f5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.331455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dce229-217f-49d6-a903-6fd2201c9375 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.331734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dce229-217f-49d6-a903-6fd2201c9375 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.332004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dce229-217f-49d6-a903-6fd2201c9375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.353616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-02dce229-217f-49d6-a903-6fd2201c9375 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:46.355732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-02dce229-217f-49d6-a903-6fd2201c9375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:46.449169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:46.452137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 106/213] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:46.453642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c25d5-7914-4892-810a-afe66ea365fb None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.461165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c25d5-7914-4892-810a-afe66ea365fb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.461165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c25d5-7914-4892-810a-afe66ea365fb None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.461165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 107/214] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:46.461165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.461790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.462256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.466772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf8236-321d-4c82-890a-bc5b5d1091f5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.484038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:46.485212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:46.504711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.505321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.505597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.505865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.506522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:46.507081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:46.508232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:46.517177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 108/215] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:46.517177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.539355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:46.559359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 108/216] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:46.560383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.639350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dff4b8-ed49-4f6c-bed4-b9a031c32c78 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.674075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780b964-373e-4d25-96c1-ce9a6b7812c3 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.674287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.674920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.675198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.675500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.676606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:46.678059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:46.684976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 109/217] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:46.684976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.709210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:46.716048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.717117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.717510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.717787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] PATH_INFO: `/v3/users/3ef02c9b383b4f7aa25ee5a1a4c6881d` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.735012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] RBAC: Authorizing `identity:get_user(user_id=3ef02c9b383b4f7aa25ee5a1a4c6881d)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:46.736953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:46.743361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 109/218] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:46 2022] GET /identity/v3/users/3ef02c9b383b4f7aa25ee5a1a4c6881d => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:46.743361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.789248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead200a-58ee-4317-99c2-eef8d039f5b5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.795816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:46.825801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.826484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.826786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.827288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.837517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:46.843647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:46.844580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 110/219] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:46.850441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:46.856741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75294276-96de-4092-8a97-24d257e52e36 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.890667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2918b8-0e07-4d82-ab1e-705516636013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.891354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2918b8-0e07-4d82-ab1e-705516636013 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.891637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2918b8-0e07-4d82-ab1e-705516636013 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.891910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2918b8-0e07-4d82-ab1e-705516636013 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:46.892576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2918b8-0e07-4d82-ab1e-705516636013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:46.894065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2918b8-0e07-4d82-ab1e-705516636013 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:46.925100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:46.942450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226d8e5-0582-4c8c-8beb-ed32f7a98385 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:46.975349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc4690-7f15-4f54-88f4-e360a42b95bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:46.975951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc4690-7f15-4f54-88f4-e360a42b95bb None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:46.976227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc4690-7f15-4f54-88f4-e360a42b95bb None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:46.976500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc4690-7f15-4f54-88f4-e360a42b95bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.033982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 110/220] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:46 2022] POST /identity/v3/users => generated 297 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:47.035552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd326378-379b-4194-a26e-7196d0ee46ea None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.035847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd326378-379b-4194-a26e-7196d0ee46ea None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.036103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd326378-379b-4194-a26e-7196d0ee46ea None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.037491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 111/221] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:47.038100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd326378-379b-4194-a26e-7196d0ee46ea None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.047007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc4690-7f15-4f54-88f4-e360a42b95bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.048537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc4690-7f15-4f54-88f4-e360a42b95bb None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.145238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:47.159950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:47.165101 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 111/222] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:47.166224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.166511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.166782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.172945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd326378-379b-4194-a26e-7196d0ee46ea None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.186299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:47.187516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:47.213778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.214471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.214757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.215025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.215764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:47.216075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.217458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.233017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 112/223] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:47.233017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.268226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 112/224] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:47.269088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.359265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65530fc7-ca23-4acc-b48f-49fd6fcd4277 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.394849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077512a6-4aa8-4762-8e3e-356d9847e4ad None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.400990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.400990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.400990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.400990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.402358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.404313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.407024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 113/225] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:47.408165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.411691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:47.439997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.440638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.440915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.441183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] PATH_INFO: `/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.442306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] RBAC: Authorizing `identity:get_user(user_id=a0e598ea6e5d4f258ff25040a0fec9f4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.443813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.445879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 113/226] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:47.446764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.447492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:47.543365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76727b8b-aee2-45b5-b10e-97a0674eb7c2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.549690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55689114-56f4-40a7-9e59-d945f6f9d39e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.580704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.581743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.582258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.582765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.584131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.584805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.585197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.585474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.586452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=3ef02c9b383b4f7aa25ee5a1a4c6881d) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:47.586629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.588148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.594553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 114/227] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/role_assignments?user.id=3ef02c9b383b4f7aa25ee5a1a4c6881d&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:47.595590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.595948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:47.601095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:47.602197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:47.603162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:47.605927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 114/228] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:47.612318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.669684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:47.757974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29026341-a8f2-4958-a49c-639ad97f6872 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.801531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.802180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.802465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.802740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.803945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.805497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.807508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 115/229] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:47.809003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:47.816402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.825657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5feeac-305b-4638-8fa9-ba70c0329b06 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:47.879753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:47.880388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:47.880752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:47.881022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:47.881694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:47.881991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:47.883416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:47.902355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 115/230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:47.915246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:47.969392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94182fc6-c378-4e46-b2b0-67f63204ad46 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:48.001241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:48.048950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:48.049928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:48.050388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:48.050863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/a0e598ea6e5d4f258ff25040a0fec9f4/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:48.056544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef07ebdea0ce49f1b264222c56073aa4, user_id=a0e598ea6e5d4f258ff25040a0fec9f4, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:48.058910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:48.076426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907ca79-b570-4248-901d-7dfea06b6fa0 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:48.112612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:48.113343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:48.113627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:48.113899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] PATH_INFO: `/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:48.115019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] RBAC: Authorizing `identity:get_user(user_id=a0e598ea6e5d4f258ff25040a0fec9f4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:48.116509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:48.118480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 116/231] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:47 2022] GET /identity/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:48.124666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:48.144220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:48.169233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 116/232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:47 2022] PUT /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/a0e598ea6e5d4f258ff25040a0fec9f4/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:48.243531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768527ad-3e50-46a2-b67e-55499adae42e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:48.290428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c4d7e-6e22-4408-9fa0-88049105ea89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:48.291068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4d7e-6e22-4408-9fa0-88049105ea89 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:48.291370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4d7e-6e22-4408-9fa0-88049105ea89 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:48.291637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c4d7e-6e22-4408-9fa0-88049105ea89 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:48.292981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4d7e-6e22-4408-9fa0-88049105ea89 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:48.294444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c4d7e-6e22-4408-9fa0-88049105ea89 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:48.296451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 117/233] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:48 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:48.302221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:48.434564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dc7d7-8aca-4f18-b92c-0aff1a5271c8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:48.476417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e9dd8-1be2-4f31-a137-f03ffcd5b00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:48.477079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9dd8-1be2-4f31-a137-f03ffcd5b00f None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:48.477420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9dd8-1be2-4f31-a137-f03ffcd5b00f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:48.477698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e9dd8-1be2-4f31-a137-f03ffcd5b00f None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/a0e598ea6e5d4f258ff25040a0fec9f4/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:48.480494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e9dd8-1be2-4f31-a137-f03ffcd5b00f None admin] RBAC: Authorizing `identity:create_grant(project_id=ef07ebdea0ce49f1b264222c56073aa4, user_id=a0e598ea6e5d4f258ff25040a0fec9f4, role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:48.482093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e9dd8-1be2-4f31-a137-f03ffcd5b00f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:48.623517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 117/234] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:48 2022] PUT /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/a0e598ea6e5d4f258ff25040a0fec9f4/roles/1cef455d27914904ba2d1176ced4233c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:49.434747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4af8fa-793a-4009-8eae-0933133a1aa4 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:49.435283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4af8fa-793a-4009-8eae-0933133a1aa4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:49.435283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4af8fa-793a-4009-8eae-0933133a1aa4 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:49.437050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 118/235] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:49.442711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:49.443116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:49.443312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:49.465048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:49.466300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:49.500250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:49.533237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583256e9-e388-4639-b2a4-3a55588aa97f None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:49.533542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583256e9-e388-4639-b2a4-3a55588aa97f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:49.533777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583256e9-e388-4639-b2a4-3a55588aa97f None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:49.535374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 119/236] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:49.540246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:49.540524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:49.540794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:49.564125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 118/237] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:49.565641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:49.567041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:49.598047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:49.641860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 120/238] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:49.669036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:49.683301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:49.778626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25bf51-0816-4bc0-931e-2455e18047a9 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:49.837113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb1606-2f0f-4572-9303-2c09524bb136 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:49.851684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:49.852612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:49.853062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:49.853514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:49.878603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:49.885171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:49.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:49.887666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:49.893206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 119/239] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:49.899254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:49.923578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:49.932469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:49.933860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:49.934142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:49.934416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:49.940174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:49.941638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:49.943282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 121/240] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:49.948755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.008357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb3de7-d324-4016-9a0c-7b1a3885e017 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.045343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:50.046268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.046714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.047164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.048050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] RBAC: Adding query filter params (name=reader) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:50.048546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:50.050335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:50.059985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 120/241] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:50.065970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.107839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:50.138807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f53829-66d5-4e13-81a2-5ddac0fa875a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.171677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:50.176464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406b206-70fc-432d-bec1-b6d86adfece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:50.177079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406b206-70fc-432d-bec1-b6d86adfece4 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.177359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406b206-70fc-432d-bec1-b6d86adfece4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.177624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406b206-70fc-432d-bec1-b6d86adfece4 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.178555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406b206-70fc-432d-bec1-b6d86adfece4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:50.180534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406b206-70fc-432d-bec1-b6d86adfece4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:50.188763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42efe6-53e3-4961-867a-0fdab228310c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.226667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03ab36-6e6e-41cb-a1b3-1a8ea55ad61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:50.227618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ab36-6e6e-41cb-a1b3-1a8ea55ad61c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.228068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ab36-6e6e-41cb-a1b3-1a8ea55ad61c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.228504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03ab36-6e6e-41cb-a1b3-1a8ea55ad61c None admin] PATH_INFO: `/v3/users/e9cceefffe5f4fe3a46169bbcade8849` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.242837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ab36-6e6e-41cb-a1b3-1a8ea55ad61c None admin] RBAC: Authorizing `identity:get_user(user_id=e9cceefffe5f4fe3a46169bbcade8849)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:50.244732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03ab36-6e6e-41cb-a1b3-1a8ea55ad61c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:50.251306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 121/242] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:50 2022] GET /identity/v3/users/e9cceefffe5f4fe3a46169bbcade8849 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:50.253739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385362e2-7a10-455e-99ce-2dba67b5fd0c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.254263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385362e2-7a10-455e-99ce-2dba67b5fd0c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.254761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385362e2-7a10-455e-99ce-2dba67b5fd0c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.256703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 122/244] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:50.257758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385362e2-7a10-455e-99ce-2dba67b5fd0c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.262697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 122/244] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:49 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:50.270895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.271625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.272123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.306318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:50.309623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:50.457316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 123/245] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:50.475472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385362e2-7a10-455e-99ce-2dba67b5fd0c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.483056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.527321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79b936-28b4-4e39-9e65-beaf08ed3495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:50.527935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79b936-28b4-4e39-9e65-beaf08ed3495 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.528210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79b936-28b4-4e39-9e65-beaf08ed3495 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.528480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79b936-28b4-4e39-9e65-beaf08ed3495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.545853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:50.569439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:50.618314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59280a-0df9-425d-9c26-3e5ddab5803d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.623449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79b936-28b4-4e39-9e65-beaf08ed3495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:50.625390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79b936-28b4-4e39-9e65-beaf08ed3495 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:50.667493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:50.668877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.669368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.669880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.687183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:50.693937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:50.694852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:50.695796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:50.699473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 124/246] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:50.705019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.763622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 123/247] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:50.764906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874f5e2-d6f0-4379-bb7c-36ef2a1fd2bc None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.765187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874f5e2-d6f0-4379-bb7c-36ef2a1fd2bc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.765562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874f5e2-d6f0-4379-bb7c-36ef2a1fd2bc None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.766917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 124/248] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:50.772476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.772762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.773132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.793503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:50.794695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:50.825081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555dac5-6141-4c80-8d6e-d2712178fa02 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.849142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:50.855163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:50.863961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:50.864576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:50.865042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:50.865157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:50.867417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:50.867417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:50.867592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:50.875699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 125/249] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:50.876567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.876676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 125/250] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:50.882092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:50.995127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9db470-1d6f-4d5d-9385-75817c796780 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:50.996810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdefef1-5550-427a-90db-987f9ee228db None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.038622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.039290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.039576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.039823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.039937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] PATH_INFO: `/v3/users/3ef02c9b383b4f7aa25ee5a1a4c6881d` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.040569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.040855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.041124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.041815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] RBAC: Authorizing `identity:get_user(user_id=3ef02c9b383b4f7aa25ee5a1a4c6881d)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.042900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.043783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.044369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.045440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 126/251] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:50 2022] GET /identity/v3/users/3ef02c9b383b4f7aa25ee5a1a4c6881d => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:51.046256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 126/252] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:50 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:51.051642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.052059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.098819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:51.115356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:51.161421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d76f1-ae26-4516-b422-6e183a5e08ca None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.163384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d60080-5eb3-4a20-97f7-3786034aff48 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.197809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.199024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.199342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.199609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.201774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.202473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.202758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.203035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.203804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] RBAC: Adding query filter params (role.id=248e2f3c1da941b8bc7e3551815d06a7, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=e9cceefffe5f4fe3a46169bbcade8849) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:51.204110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.205549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.210818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:51.212048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 127/253] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/role_assignments?user.id=e9cceefffe5f4fe3a46169bbcade8849&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=248e2f3c1da941b8bc7e3551815d06a7 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:51.212650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:51.216863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:51.218983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:51.220918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 127/254] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:51.228351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.372755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b30dc8-ba26-4a02-9eb3-6eedffc1459d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.375546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:51.413825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc0fb8-8cca-45ca-9437-736448b1041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.414708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc0fb8-8cca-45ca-9437-736448b1041b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.415156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc0fb8-8cca-45ca-9437-736448b1041b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.415593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc0fb8-8cca-45ca-9437-736448b1041b None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.416973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc0fb8-8cca-45ca-9437-736448b1041b None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.418703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc0fb8-8cca-45ca-9437-736448b1041b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.420620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 128/255] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:51.422133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5903c-4658-4efa-8b5f-85b65b25a4da None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.422548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5903c-4658-4efa-8b5f-85b65b25a4da None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.422941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad5903c-4658-4efa-8b5f-85b65b25a4da None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.425894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 129/256] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:51.426814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5903c-4658-4efa-8b5f-85b65b25a4da None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.482947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0296d-29d3-4213-a498-f285d83de85b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.516610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:51.545041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad5903c-4658-4efa-8b5f-85b65b25a4da None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.554256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.554615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.555887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.556282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.557066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:51.557520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.559456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15c3f3-d574-4b98-b91e-e5328a6df83a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.578987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 128/257] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:51.580455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.580855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.581142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.597990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.598965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.599521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.599970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/3ef02c9b383b4f7aa25ee5a1a4c6881d/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.603677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] RBAC: Authorizing `identity:create_grant(project_id=ef07ebdea0ce49f1b264222c56073aa4, user_id=3ef02c9b383b4f7aa25ee5a1a4c6881d, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.605759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.611390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:51.612885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:51.677845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 130/258] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:51 2022] PUT /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/3ef02c9b383b4f7aa25ee5a1a4c6881d/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:51.678537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.712641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 129/259] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:51.738573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.749890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:51.852439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8aaee-b52b-42c6-ace8-8507f917256b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:51.926658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:51.927309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:51.927587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:51.927898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] PATH_INFO: `/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:51.931707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] RBAC: Authorizing `identity:get_user(user_id=a0e598ea6e5d4f258ff25040a0fec9f4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:51.933796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:51.935984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 131/260] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:51.941263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:51.992904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd98b56-e38b-46d5-b6e3-8eb874a6f252 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:52.019093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:52.069524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:52.070168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:52.070447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:52.074786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:52.098081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:52.103100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:52.104035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:52.105560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 130/261] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:52.111702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:52.136670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:52.144895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e07764-9bd0-4c39-ba3e-41cddd26772c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:52.217962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:52.218501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:52.218779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:52.219049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:52.301776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb9ad-faf0-4633-8285-fbc3af3c19d8 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:52.345306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:52.346778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:52.368057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:52.369027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:52.369502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:52.369966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:52.370910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:52.371432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:52.373155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:52.389876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 131/262] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:52.395919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:52.430512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:52.514817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6b6b3-dcda-4c12-ad76-f5a352de1c01 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:52.532619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:52.538973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 132/263] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:52.544902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:52.588069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:52.591933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:52.592427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:52.592897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] PATH_INFO: `/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:52.594300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] RBAC: Authorizing `identity:get_user(user_id=a0e598ea6e5d4f258ff25040a0fec9f4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:52.598336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:52.601342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 132/264] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:52 2022] GET /identity/v3/users/a0e598ea6e5d4f258ff25040a0fec9f4 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:52.607162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:52.741987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f54915-a288-4cf6-8e28-f76e4c44db88 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:52.802175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:52.806951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da669d2-14c0-4fcd-9cc3-3e47d5c23885 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:52.844488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:52.845130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:52.845399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:52.845665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:52.851902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:52.853356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:52.855927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 133/265] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:52 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:52.861325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:52.884673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:52.885922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:52.887052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:52.887630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.008133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:53.031147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:53.036757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:53.090727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db39e3c-4af2-4b44-8d42-b68e6f41d9d3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:53.184500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:53.185120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.185398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.185667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.186391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=a0e598ea6e5d4f258ff25040a0fec9f4) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:53.186692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:53.188148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:53.190512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:53.194681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 133/266] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:53.200749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:53.205598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 134/267] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:52 2022] GET /identity/v3/role_assignments?user.id=a0e598ea6e5d4f258ff25040a0fec9f4&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:53.356990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21506483-2b6f-478f-9ce8-8b1a60b90980 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:53.414763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cbf34c-82e4-4a68-bd22-6fe531c32b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:53.415445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbf34c-82e4-4a68-bd22-6fe531c32b86 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.415726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbf34c-82e4-4a68-bd22-6fe531c32b86 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.415996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cbf34c-82e4-4a68-bd22-6fe531c32b86 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.417236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbf34c-82e4-4a68-bd22-6fe531c32b86 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:53.418710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cbf34c-82e4-4a68-bd22-6fe531c32b86 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:53.420705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 134/268] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:53 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:53.426535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:53.435556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201230f-dd3c-453f-88b7-0344802c5c87 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.435741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201230f-dd3c-453f-88b7-0344802c5c87 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.436013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6201230f-dd3c-453f-88b7-0344802c5c87 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.437367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 135/269] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:53.450266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.450592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.450817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.484372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:53.485604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:53.568713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:53.579980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 136/270] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:53.609926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:53.655478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c64e3b-98f9-4e73-a9f2-b59c216da9e0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:53.724096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:53.732413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:53.733030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.733305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.733586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.734306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] RBAC: Adding query filter params (role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=a0e598ea6e5d4f258ff25040a0fec9f4) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:53.734611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:53.736083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb04b8-c20b-433b-9886-d23ca84a4666 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:53.751744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 135/271] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:53 2022] GET /identity/v3/role_assignments?user.id=a0e598ea6e5d4f258ff25040a0fec9f4&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=1cef455d27914904ba2d1176ced4233c => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:53.759098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873321e7-4431-4438-8d09-868ac6b2c13e None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.759924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873321e7-4431-4438-8d09-868ac6b2c13e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.760546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873321e7-4431-4438-8d09-868ac6b2c13e None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.762650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 136/272] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:53.769425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.770020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.770605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.787211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c121601-d1bc-49e1-8f20-f8f0848bd38c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:53.793082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:53.794727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:53.852331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:53.852708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:53.852986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:53.853253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:53.869428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:53.874523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:53.875460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:53.876256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:53.879741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 137/273] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:53.879916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 137/274] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:53.887923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:53.913853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:53.978357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.033102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6f23c-88ef-42dc-9a37-c0bbe16872cf None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.035204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4e826b-d70b-49ff-936c-9b2cd5908647 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.063617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.074769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.075892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.076372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.077587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.078428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:54.080154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:54.082373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 138/275] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:54.088495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.136187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.137029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.137248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.137527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.138200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:54.138500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:54.139999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:54.150450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 138/276] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:54.158863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.224859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e0315b-c6b0-406f-8580-bce9b682809e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.267383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5d2cd-eac5-405e-84c4-560df9afea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.268511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5d2cd-eac5-405e-84c4-560df9afea78 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.268794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5d2cd-eac5-405e-84c4-560df9afea78 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.269064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5d2cd-eac5-405e-84c4-560df9afea78 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.269706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5d2cd-eac5-405e-84c4-560df9afea78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:54.271144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5d2cd-eac5-405e-84c4-560df9afea78 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:54.290625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d3991-ec7a-4cf4-96d3-86a708373d98 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.312009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.325588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaef7c-a080-4b57-8c98-38bc69e36164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.326213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaef7c-a080-4b57-8c98-38bc69e36164 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.326489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaef7c-a080-4b57-8c98-38bc69e36164 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.326759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaef7c-a080-4b57-8c98-38bc69e36164 None admin] PATH_INFO: `/v3/users/a9228284273d4658b1414d335c97d44a` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.340359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaef7c-a080-4b57-8c98-38bc69e36164 None admin] RBAC: Authorizing `identity:get_user(user_id=a9228284273d4658b1414d335c97d44a)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:54.341811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaef7c-a080-4b57-8c98-38bc69e36164 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:54.343443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 139/277] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/users/a9228284273d4658b1414d335c97d44a => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:54.344575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671dc7-76ff-4de0-a3ee-50e826aa5808 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.344850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671dc7-76ff-4de0-a3ee-50e826aa5808 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.345120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad671dc7-76ff-4de0-a3ee-50e826aa5808 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.346310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 140/278] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:54.347328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09492b0f-218e-431c-8d9e-dcebe06eb3c1 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.347606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09492b0f-218e-431c-8d9e-dcebe06eb3c1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.353317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09492b0f-218e-431c-8d9e-dcebe06eb3c1 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.353317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 141/279] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:54.353317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09492b0f-218e-431c-8d9e-dcebe06eb3c1 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.381750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 139/280] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:13:54 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:13:54.381750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8bc71-daf0-4050-b09d-03743c837f2f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.381750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8bc71-daf0-4050-b09d-03743c837f2f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.381750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8bc71-daf0-4050-b09d-03743c837f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.400195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-e5e8bc71-daf0-4050-b09d-03743c837f2f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:54.401920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e5e8bc71-daf0-4050-b09d-03743c837f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:54.455184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.474217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09492b0f-218e-431c-8d9e-dcebe06eb3c1 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.491261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 140/281] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:54.491261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.491261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.491261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.508892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:54.510492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:54.524388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.524997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.525272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.525538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.526791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, user.id=a9228284273d4658b1414d335c97d44a, scope.system=all) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:54.527568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:54.528571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:54.535260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 142/282] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/role_assignments?user.id=a9228284273d4658b1414d335c97d44a&scope.system=all&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:54.536058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.594934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.609463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 141/283] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:54.632571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.677348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444cdcc-0616-43b1-ac8f-5fe969e85805 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.725524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.726159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.726432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.726698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.743649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:54.750613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:54.751757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:54.752741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 143/284] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:54.759516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.769451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbad9e-8046-4dda-8ff1-962b09a955b6 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.783458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.815777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.817183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.817633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.818095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.832633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:54.839998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:54.841099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:54.842089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:54.844896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 142/285] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:54.850864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.888552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a7522-fbcf-4cb8-ae1a-29ec0051abc4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.891548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:54.926255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.927007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:54.928020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:54.928020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:54.928564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] RBAC: Adding query filter params (name=reader) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:54.928965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:54.930499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:54.940960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 144/286] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:54.946395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:54.961707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff770e6-a7fe-45a8-a7ef-80053a4afac2 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:54.998742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:54.999639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.000060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.000472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.001298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:55.001744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.003293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.012625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 143/287] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:55.018396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:55.053064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:55.057309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb710337-ffd8-4872-a4e6-d6d64053a81f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:55.098584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:55.099417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.099801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.100173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] PATH_INFO: `/v3/users/e9cceefffe5f4fe3a46169bbcade8849` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.101822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] RBAC: Authorizing `identity:get_user(user_id=e9cceefffe5f4fe3a46169bbcade8849)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.103417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.106444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 145/288] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:54 2022] GET /identity/v3/users/e9cceefffe5f4fe3a46169bbcade8849 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:55.111891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:55.143080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd02faa6-68b2-47b0-86e0-9418c1041114 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:55.145647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:55.186623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:55.187511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.188092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.188506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] PATH_INFO: `/v3/users/3ef02c9b383b4f7aa25ee5a1a4c6881d` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.190850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] RBAC: Authorizing `identity:get_user(user_id=3ef02c9b383b4f7aa25ee5a1a4c6881d)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.192468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.194905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 144/289] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:55 2022] GET /identity/v3/users/3ef02c9b383b4f7aa25ee5a1a4c6881d => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:55.200967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:55.236935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26db6e9-1e78-4553-959f-92039435e5eb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:55.302961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:55.302961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.302961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.302961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.304274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.306024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.308447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 146/290] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:55 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:55.317115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:55.339924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:55.343257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f2858-f41e-413d-b8f4-b67ac655f01c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:55.402295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:55.403143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.403596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.403971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.508086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2086b7-8aa2-406c-9c5e-1ee53580d7ac None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:55.511623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:55.538430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.540382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.592519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:55.594391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.594816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.596046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/e9cceefffe5f4fe3a46169bbcade8849/roles/248e2f3c1da941b8bc7e3551815d06a7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.599107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] RBAC: Authorizing `identity:create_grant(project_id=ef07ebdea0ce49f1b264222c56073aa4, user_id=e9cceefffe5f4fe3a46169bbcade8849, role_id=248e2f3c1da941b8bc7e3551815d06a7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.602743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.661341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:55.668371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 145/291] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:13:55.675704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 147/292] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:13:55 2022] PUT /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/users/e9cceefffe5f4fe3a46169bbcade8849/roles/248e2f3c1da941b8bc7e3551815d06a7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:55.677001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:55.825119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f121d36d-3502-4583-a7e1-75bed4455f93 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:55.863776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f5d23-a17b-4e80-bd09-45b237f6bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:55.864435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5d23-a17b-4e80-bd09-45b237f6bd72 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:55.864753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5d23-a17b-4e80-bd09-45b237f6bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:55.865053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f5d23-a17b-4e80-bd09-45b237f6bd72 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:55.866231 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5d23-a17b-4e80-bd09-45b237f6bd72 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:55.868001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f5d23-a17b-4e80-bd09-45b237f6bd72 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:55.870620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 146/293] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:55 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:55.876565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:56.000690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74212f7e-44cd-49bb-b712-346320f2bd4d None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:56.040230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:56.040863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:56.041196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:56.041475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:56.042313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=3ef02c9b383b4f7aa25ee5a1a4c6881d) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:56.042713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:56.044349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728ee5d1-4e10-4022-b580-18eddfef4c4f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:56.051335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 148/294] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:55 2022] GET /identity/v3/role_assignments?user.id=3ef02c9b383b4f7aa25ee5a1a4c6881d&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:56.737077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee081993-7198-40cc-8389-425e1c3b62e0 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:56.737426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee081993-7198-40cc-8389-425e1c3b62e0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:56.737689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee081993-7198-40cc-8389-425e1c3b62e0 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:56.739338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 147/295] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:56.744640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:56.745124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:56.745301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:56.766905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:56.768134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:56.790848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:56.844306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 149/296] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:56.866538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee081993-7198-40cc-8389-425e1c3b62e0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:56.938348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:56.974455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee081993-7198-40cc-8389-425e1c3b62e0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.011438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d82966-15c2-42bf-930f-c79afbb2bade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.012152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d82966-15c2-42bf-930f-c79afbb2bade None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.012427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d82966-15c2-42bf-930f-c79afbb2bade None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.012692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d82966-15c2-42bf-930f-c79afbb2bade None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.023535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d82966-15c2-42bf-930f-c79afbb2bade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e1d82966-15c2-42bf-930f-c79afbb2bade None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:57.029059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:57.029976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:57.030771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 148/297] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:57.036386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.121395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92021b19-f515-4c60-9da5-50d8ebaf6e55 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.121725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92021b19-f515-4c60-9da5-50d8ebaf6e55 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.121985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92021b19-f515-4c60-9da5-50d8ebaf6e55 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.123625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 149/298] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:57.128623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af2b9-399d-4ef3-9cde-6d7639d02537 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.128899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af2b9-399d-4ef3-9cde-6d7639d02537 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.129166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448af2b9-399d-4ef3-9cde-6d7639d02537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.149196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-448af2b9-399d-4ef3-9cde-6d7639d02537 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:57.150374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-448af2b9-399d-4ef3-9cde-6d7639d02537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:57.152689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceb0b5-21aa-48cb-9118-be3b8365cfbd None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.190865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.191435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.191684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.191958 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.192621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:57.192919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:57.194292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb729e6-709c-464e-904d-50f9489d6f4d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:57.203304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 150/299] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:57.204501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188413-f576-474c-9e5b-d7ea336555b5 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.204783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188413-f576-474c-9e5b-d7ea336555b5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.205054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e188413-f576-474c-9e5b-d7ea336555b5 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.206417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 151/300] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:57.209124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e188413-f576-474c-9e5b-d7ea336555b5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.229277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:57.235964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 150/301] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:57.237051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.237340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.237615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.255963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:57.257211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:57.314298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:57.321549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e188413-f576-474c-9e5b-d7ea336555b5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.335253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 151/302] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:57.335253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.364937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.365569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.365859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.366122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] PATH_INFO: `/v3/users/a9228284273d4658b1414d335c97d44a` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.367219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] RBAC: Authorizing `identity:get_user(user_id=a9228284273d4658b1414d335c97d44a)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:57.368680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:57.370777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 152/303] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:57 2022] GET /identity/v3/users/a9228284273d4658b1414d335c97d44a => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:57.371674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.456402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e80a64-993e-4356-8700-46fc8f9dc25d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.462854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:57.478943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f890e-8817-4dc1-8b6d-20a1f16eca18 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.491602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.492229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.492504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.492780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.493858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:57.495959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:57.497675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 152/304] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:57.500049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.519394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.520023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.520300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.520571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.531830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:57.537108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:57.538073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:13:57.538878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 153/305] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:13:57.539563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.570136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:57.621047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37563b8f-4f82-4ea7-92e5-118ba5cb889d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.644389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92179098-bb1e-48c6-9c25-37b3a35c7266 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.663710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.664777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.665055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.665474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] PATH_INFO: `/v3/system/users/a9228284273d4658b1414d335c97d44a/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.669149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a9228284273d4658b1414d335c97d44a, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:57.669792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:57.679344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.679957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.680233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.680501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] PATH_INFO: `/v3/groups` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.681115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:57.683337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:57.766571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 153/306] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Aug 8 00:13:57 2022] PUT /identity/v3/system/users/a9228284273d4658b1414d335c97d44a/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:13:57.767193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.831256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 154/307] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Aug 8 00:13:57 2022] POST /identity/v3/groups => generated 229 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:13:57.901641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c294eb-659d-4c6f-b8a1-291aed850224 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:57.927016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:57.948817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:57.949650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:57.949927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:57.950191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:57.950872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:57.951172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:57.952615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d43b0-0681-47d7-9005-8e49ea81b0d9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:57.962867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 154/308] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:57.969500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:57.975213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:58.085700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb250b-ad8b-44d2-9aec-748e113da474 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.085988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb250b-ad8b-44d2-9aec-748e113da474 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.086243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb250b-ad8b-44d2-9aec-748e113da474 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.086390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d527a66-745d-4d05-9340-e12a60e094c9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:58.087886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 155/309] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:13:58.093094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.093356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.093629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.113512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:58.114695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:58.124090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:58.124784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.125164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.125438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] PATH_INFO: `/v3/users/5091517659e7432db578a80e0df5ba13` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.140803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] RBAC: Authorizing `identity:get_user(user_id=5091517659e7432db578a80e0df5ba13)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:58.142350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:58.144447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 155/310] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:57 2022] GET /identity/v3/users/5091517659e7432db578a80e0df5ba13 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:58.150291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:58.192104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 156/311] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:13:58.216328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:58.239653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:58.274400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29919dee-689f-4b46-a657-7de938fe5f76 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:58.309728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:58.310329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.310602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.310872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.311694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] RBAC: Adding query filter params (role.id=248e2f3c1da941b8bc7e3551815d06a7, user.id=5091517659e7432db578a80e0df5ba13, scope.system=all) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:58.311997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:58.313423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:58.319277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 156/312] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 00:13:58 2022] GET /identity/v3/role_assignments?user.id=5091517659e7432db578a80e0df5ba13&scope.system=all&role.id=248e2f3c1da941b8bc7e3551815d06a7 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:13:58.334715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192e13f-0c2b-4d5c-ba2a-2372afa32d51 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:58.351656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:58.375542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:58.376199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.376478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.376751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.389717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:13:58.395037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:13:58.396058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:13:58.396842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:13:58.403622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 157/313] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:13:58.406683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:58.517372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0381229-603a-428b-a993-60a825cc55a1 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:58.544169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:58.552559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:58.553169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.553440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.553709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.554369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:58.554665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:58.556240 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:58.566422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 157/314] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:13:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:58.572133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:58.691422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c5497-f6a0-4870-821d-21d02f68e06e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:58.726372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:58.727388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.727887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.728378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] PATH_INFO: `/v3/users/e9cceefffe5f4fe3a46169bbcade8849` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.730307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] RBAC: Authorizing `identity:get_user(user_id=e9cceefffe5f4fe3a46169bbcade8849)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:58.731955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:58.734030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 158/315] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:13:58 2022] GET /identity/v3/users/e9cceefffe5f4fe3a46169bbcade8849 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:58.739827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:58.845909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011b9ec-984d-4bde-99a0-f1733b7ef802 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:58.881999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:58.882651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:58.882930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:58.883239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:58.942507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:58.956654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:58.958182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:59.065251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 158/316] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:13:59.071043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:59.116619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:59.179549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160b810-fa31-4d49-9a1f-17e618232a0e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:59.217831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8a82b-144d-49a6-9208-f8a6eb0110ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:59.217831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8a82b-144d-49a6-9208-f8a6eb0110ce None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:59.217831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8a82b-144d-49a6-9208-f8a6eb0110ce None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:59.217831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8a82b-144d-49a6-9208-f8a6eb0110ce None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:59.219144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8a82b-144d-49a6-9208-f8a6eb0110ce None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:59.220889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc8a82b-144d-49a6-9208-f8a6eb0110ce None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:59.223469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 159/317] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:13:59 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:59.230337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:13:59.320052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:13:59.332769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563ce64-4515-4186-abe2-d8fe0b5cf674 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:13:59.366968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:13:59.367597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:59.367872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:59.368140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:59.368861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] RBAC: Adding query filter params (role.id=248e2f3c1da941b8bc7e3551815d06a7, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4, user.id=e9cceefffe5f4fe3a46169bbcade8849) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:13:59.369163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:13:59.371342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d290b-39e5-4472-9807-14beb2c1cd9d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:13:59.377545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 159/318] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:13:59 2022] GET /identity/v3/role_assignments?user.id=e9cceefffe5f4fe3a46169bbcade8849&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=248e2f3c1da941b8bc7e3551815d06a7 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:13:59.873776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1457e15-7403-4b19-8cf4-411724299d87 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:59.874148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1457e15-7403-4b19-8cf4-411724299d87 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:59.874325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1457e15-7403-4b19-8cf4-411724299d87 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:59.875843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 160/319] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:59.880842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f58a1-f472-4eb7-a013-7b717c598c62 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:59.881283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f58a1-f472-4eb7-a013-7b717c598c62 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:59.881444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f58a1-f472-4eb7-a013-7b717c598c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:59.899620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-180f58a1-f472-4eb7-a013-7b717c598c62 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:13:59.900863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-180f58a1-f472-4eb7-a013-7b717c598c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:13:59.968753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 160/320] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:13:59.978126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae486e-d6b5-4466-a6d5-4c7a0899d12c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:59.978408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae486e-d6b5-4466-a6d5-4c7a0899d12c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:59.978677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae486e-d6b5-4466-a6d5-4c7a0899d12c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:59.980671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 161/321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:13:59.985539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:13:59.985826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:13:59.986102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:13:59.991979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae486e-d6b5-4466-a6d5-4c7a0899d12c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.006890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:00.008390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:00.077471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 161/322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:00.099467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.102983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae486e-d6b5-4466-a6d5-4c7a0899d12c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.113767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:00.114262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:00.139305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.139928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.140201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.140467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.151697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:00.156738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:00.162275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 162/323] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:13:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:00.163830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.209734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43030c6-61c6-40d7-aca4-e80601cab004 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.243902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.244453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.244730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.245652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.246822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:00.248299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:00.250037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 162/324] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:00.256180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.273258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9df1b6-a70d-47d0-ad57-6094c69e6460 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.308321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.308965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.309241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.309509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.310165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:00.310469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:00.311869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:00.320810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 163/325] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:00.326854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.341399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:00.359437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:00.360814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b88f0-a86f-4421-a1b3-9681fd7c4498 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.397686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e71474-097d-4165-9df3-ab2aa4545889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.398286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e71474-097d-4165-9df3-ab2aa4545889 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.398558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e71474-097d-4165-9df3-ab2aa4545889 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.398833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e71474-097d-4165-9df3-ab2aa4545889 None admin] PATH_INFO: `/v3/groups` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.399475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e71474-097d-4165-9df3-ab2aa4545889 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:00.400942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e71474-097d-4165-9df3-ab2aa4545889 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:00.431086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcec4716-fb7e-4f0f-bc77-d0939dc91b87 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.464307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4344e-84d1-45c4-9f6e-47eb966b6584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.464927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4344e-84d1-45c4-9f6e-47eb966b6584 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.465204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4344e-84d1-45c4-9f6e-47eb966b6584 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.465476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4344e-84d1-45c4-9f6e-47eb966b6584 None admin] PATH_INFO: `/v3/users/a9228284273d4658b1414d335c97d44a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.466641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4344e-84d1-45c4-9f6e-47eb966b6584 None admin] RBAC: Authorizing `identity:get_user(user_id=a9228284273d4658b1414d335c97d44a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:00.468088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a4344e-84d1-45c4-9f6e-47eb966b6584 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:00.469637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 164/326] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/users/a9228284273d4658b1414d335c97d44a => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:00.470717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fa9f3-b372-4a35-a9ad-94150c5fe896 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.470994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fa9f3-b372-4a35-a9ad-94150c5fe896 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.471281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996fa9f3-b372-4a35-a9ad-94150c5fe896 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.472470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 165/327] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:00.475253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fa9f3-b372-4a35-a9ad-94150c5fe896 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.502017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 163/328] 10.222.0.15 () {60 vars in 1134 bytes} [Mon Aug 8 00:14:00 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:14:00.502017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.502017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.502017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.520281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:00.521515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:00.582455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:00.587030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996fa9f3-b372-4a35-a9ad-94150c5fe896 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.604079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 164/329] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:00.625713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.626337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.626613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.626885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.627747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] RBAC: Adding query filter params (role.id=094f7bfb155b40fea515d8a0935ab3db, user.id=a9228284273d4658b1414d335c97d44a, scope.system=all) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:00.628055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:00.628316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.629534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:00.644210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 166/330] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/role_assignments?user.id=a9228284273d4658b1414d335c97d44a&scope.system=all&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:00.719404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:00.752505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bcdb2-7e3d-4576-ad12-07bcb5b055ca None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.793782 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.794432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.794709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.794975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.806576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:00.812154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:00.813023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:00.813804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 165/331] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:00.819553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:00.926392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a73c3-858c-46f1-8b9c-03ae49830d26 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:00.962092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:00.962725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:00.963025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:00.963322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:00.964015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] RBAC: Adding query filter params (name=reader) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:00.964314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:00.965803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:00.979770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 167/332] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:00.983330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:01.092530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceb03a-53f2-4069-8397-a1be17b185fc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:01.126038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:01.132881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d736e1-3e6d-48ea-a8c7-78f78fcb972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:01.133516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d736e1-3e6d-48ea-a8c7-78f78fcb972e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:01.133816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d736e1-3e6d-48ea-a8c7-78f78fcb972e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:01.134057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d736e1-3e6d-48ea-a8c7-78f78fcb972e None admin] PATH_INFO: `/v3/users/5091517659e7432db578a80e0df5ba13` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:01.135331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d736e1-3e6d-48ea-a8c7-78f78fcb972e None admin] RBAC: Authorizing `identity:get_user(user_id=5091517659e7432db578a80e0df5ba13)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:01.137580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d736e1-3e6d-48ea-a8c7-78f78fcb972e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:01.144296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 166/333] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:14:00 2022] GET /identity/v3/users/5091517659e7432db578a80e0df5ba13 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:01.145604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:01.168483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:01.260363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7973af-fe59-4bf4-b4d8-d3355a4f034b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:01.296932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38a73d-c01f-4124-aa45-378f87c1bd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:01.297601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38a73d-c01f-4124-aa45-378f87c1bd81 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:01.297881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38a73d-c01f-4124-aa45-378f87c1bd81 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:01.298155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38a73d-c01f-4124-aa45-378f87c1bd81 None admin] PATH_INFO: `/v3/system/users/5091517659e7432db578a80e0df5ba13/roles/248e2f3c1da941b8bc7e3551815d06a7` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:01.299624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38a73d-c01f-4124-aa45-378f87c1bd81 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5091517659e7432db578a80e0df5ba13, role_id=248e2f3c1da941b8bc7e3551815d06a7)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:01.301008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38a73d-c01f-4124-aa45-378f87c1bd81 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:01.364215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 168/334] 10.222.0.15 () {58 vars in 1334 bytes} [Mon Aug 8 00:14:01 2022] PUT /identity/v3/system/users/5091517659e7432db578a80e0df5ba13/roles/248e2f3c1da941b8bc7e3551815d06a7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:02.933403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb37b-e820-43d8-b499-d04dfff66e79 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:02.933881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb37b-e820-43d8-b499-d04dfff66e79 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:02.933881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cb37b-e820-43d8-b499-d04dfff66e79 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:02.935473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 167/335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:02.941041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb4f03-a3e3-4a52-a528-8c3da1cc506b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:02.941442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb4f03-a3e3-4a52-a528-8c3da1cc506b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:02.941611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb4f03-a3e3-4a52-a528-8c3da1cc506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:02.963363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1dcb4f03-a3e3-4a52-a528-8c3da1cc506b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:02.964617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1dcb4f03-a3e3-4a52-a528-8c3da1cc506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:03.045312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 169/336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:03.081279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb37b-e820-43d8-b499-d04dfff66e79 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:03.103923 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39071881-155b-433b-98a0-f984f1337540 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.104288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39071881-155b-433b-98a0-f984f1337540 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.104572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39071881-155b-433b-98a0-f984f1337540 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.106156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 170/337] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:03.120925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.121204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.121482 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.159373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:03.160588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:03.168705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:03.242717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cb37b-e820-43d8-b499-d04dfff66e79 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:03.317197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:03.317837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.318108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.318373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.321767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 171/338] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:03.342627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:03.347435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:03.348310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:03.349127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 168/339] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:03.355302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:03.358851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:03.468176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:03.551466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1559bf-2171-4795-8359-3e1c9342083b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:03.594943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac053ddd-fa5c-4bb3-8cf4-d906aea84607 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:03.611933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:03.612577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.612848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.613522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.639847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:03.645278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:03.646158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:03.646951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:03.652950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 169/340] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:03.658704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:03.676897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:03.677528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.677813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.678085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.678762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:03.679059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:03.680460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45200a1c-6e9f-4ac5-83b1-cecac15d58dd None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:03.694383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 172/341] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:03.702341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab90b0c-a1d0-42a4-b986-2aef853ad7d3 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.702630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab90b0c-a1d0-42a4-b986-2aef853ad7d3 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.702910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab90b0c-a1d0-42a4-b986-2aef853ad7d3 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.704516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 173/342] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:03.706371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab90b0c-a1d0-42a4-b986-2aef853ad7d3 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:03.716416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:03.797606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a6c8f-43c9-4289-9073-bec7b85712a2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:03.821062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:03.847489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:03.848104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.848378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.848648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.849370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:03.849668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:03.851260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94c9bb-f6a1-4518-8dc8-2b3c2cc78c35 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:03.865164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 170/343] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:03.866480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.866767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.867043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.884017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab90b0c-a1d0-42a4-b986-2aef853ad7d3 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:03.891718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:03.892902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:03.927540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:03.928148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:03.928421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:03.928689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:03.982704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 171/344] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:03.983673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:04.282532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:04.284706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:04.294911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:04.338144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fdfb5-ddb8-4dd4-9b32-00ec4f4b48cc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:04.355015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:04.392710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0d9b1-9bf6-439b-96ab-cb9d34a2c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:04.393719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d9b1-9bf6-439b-96ab-cb9d34a2c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:04.394303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d9b1-9bf6-439b-96ab-cb9d34a2c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:04.394780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d9b1-9bf6-439b-96ab-cb9d34a2c6a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:04.442118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 174/345] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:04.443107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:04.501518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0d9b1-9bf6-439b-96ab-cb9d34a2c6a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:04.509171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0d9b1-9bf6-439b-96ab-cb9d34a2c6a1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:04.556901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93e857-6953-4ae0-bbf2-46e06a64189e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:04.559983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:04.611528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:04.612169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:04.612456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:04.612729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:04.638913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:04.644225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:04.645292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:04.646138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 175/346] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:04.646849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:04.688206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 172/347] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:04.689901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c5da-daed-4c1c-9f1d-e7c722278c7e None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:04.690849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c5da-daed-4c1c-9f1d-e7c722278c7e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:04.691198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408c5da-daed-4c1c-9f1d-e7c722278c7e None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:04.696933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 173/348] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:04.697570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408c5da-daed-4c1c-9f1d-e7c722278c7e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:04.721493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:04.794512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1ef93-1b7b-483f-83a2-a34253fd5e49 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:04.842310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:04.845365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:04.845365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:04.845580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408c5da-daed-4c1c-9f1d-e7c722278c7e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:04.845674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:04.845674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:04.846563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:04.848111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 176/349] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:04 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:04.848902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:04.891812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:04.896847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:04.897475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:04.897755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:04.898028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:04.898693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:04.898994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:04.900428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13c872-ab96-4d59-b26d-d932bdb1a1a9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:04.911422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 174/350] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:04.912665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6376b01-68ab-407d-95e8-811dc24b7059 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:04.913004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6376b01-68ab-407d-95e8-811dc24b7059 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:04.913277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6376b01-68ab-407d-95e8-811dc24b7059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:04.938216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c6376b01-68ab-407d-95e8-811dc24b7059 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:04.939939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c6376b01-68ab-407d-95e8-811dc24b7059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:04.977395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6927f0-75d9-4931-94b1-3e0397a17f22 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.017505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 175/351] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:05.018645 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f09d5-afe6-4fef-814a-60a12a7d848a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.018925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f09d5-afe6-4fef-814a-60a12a7d848a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.019203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f09d5-afe6-4fef-814a-60a12a7d848a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.019930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.020761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.020878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 176/352] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:05.021313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.021526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f09d5-afe6-4fef-814a-60a12a7d848a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.021907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.023175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:05.024311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.024637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:05.026993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 177/353] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:04 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:05.031910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.151321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f09d5-afe6-4fef-814a-60a12a7d848a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.151498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.154362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b0c66-ffad-4051-ae8a-2128b1029daf None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.190368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8796884-1c73-4bd2-a5e5-e4062a08aa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.191002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8796884-1c73-4bd2-a5e5-e4062a08aa86 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.191300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8796884-1c73-4bd2-a5e5-e4062a08aa86 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.191574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8796884-1c73-4bd2-a5e5-e4062a08aa86 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.191956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.192558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.192838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.193106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.198798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8796884-1c73-4bd2-a5e5-e4062a08aa86 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:05.200306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8796884-1c73-4bd2-a5e5-e4062a08aa86 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:05.202424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 177/354] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:05.203694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c97d-65a8-495c-8031-59faf19c79c3 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.204867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c97d-65a8-495c-8031-59faf19c79c3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.204867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c97d-65a8-495c-8031-59faf19c79c3 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.205427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 178/355] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:05.206416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.206688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.209159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.225998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:05.227202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:05.285347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:05.286797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:05.296602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.307332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 179/356] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:05.311203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.403739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.411405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 178/357] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:05.412769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.426503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805aa96c-ae6f-4679-a01d-e169c71ed30e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.469620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.470314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.470597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.470868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.472197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:05.473501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:05.475273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 180/358] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:05.475868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.535464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabba669-7de6-4921-9ba0-fd4f0d4a0f90 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.552083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.576812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1ae81-84bc-4ae1-9be8-fd1f4884ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.577455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1ae81-84bc-4ae1-9be8-fd1f4884ba00 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.577732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1ae81-84bc-4ae1-9be8-fd1f4884ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.577999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1ae81-84bc-4ae1-9be8-fd1f4884ba00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.589307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c1ae81-84bc-4ae1-9be8-fd1f4884ba00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a3c1ae81-84bc-4ae1-9be8-fd1f4884ba00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:05.594164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:05.595364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:05.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 179/359] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:05.597212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.597212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.597212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.600275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef1c7-ae8d-4738-b536-58261b8a5c8b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.615461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:05.617026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:05.641829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.642439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.642719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.642990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.643748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:05.644058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:05.645514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:05.654895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 181/360] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:05.656737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.700372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.715347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 180/361] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:05.716250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.803783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c70a8-f4b8-4c90-a9f6-56bae62600db None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.842637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:05.849526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.850176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.850440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.850709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.856888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbce1fe-9f7b-428a-9a0a-30a935e2e427 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:05.868218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:05.875270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:05.876349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:05.877298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:05.879337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 182/362] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:05.880273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.900713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:05.901549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:05.901922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:05.902306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:05.903696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:05.905192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:05.907027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 181/363] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:05.907646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:05.979037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.004583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31f9851-1175-4760-9ec1-edc7489c8c42 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.028336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3ca1-844d-4e73-83c4-650d7562c737 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.046149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.046760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.047034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.047338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.048061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:06.048366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.049794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.056692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 183/364] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=1cef455d27914904ba2d1176ced4233c => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.058090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.077410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.078252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.078680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.079047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.079942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] RBAC: Adding query filter params (name=reader) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:06.080346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.082187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.093314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 182/365] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.094422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.136202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.196536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22d033-2f14-41a4-b5ff-c1ddf7cff63a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.242710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.243696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.244073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.244484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.246924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282fb8-db7b-4357-b458-a1c7b6b5f82c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.258688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:06.264921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:06.265808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:06.266610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 184/366] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:06.268830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.269454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.289943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.290723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.291079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.291508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.293007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:06.293572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.295490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.311136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 183/367] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.312224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.420775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895cdae-29e0-4c98-9cb5-e72ef0f55e03 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.439363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.441316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43401c8d-497d-43c6-9f8e-c0c3173c3162 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.468354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.469493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.470181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.470623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.472603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.475173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.477249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 185/368] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.478192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.483883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.484663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.485010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.485351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] PATH_INFO: `/v3/users/5091517659e7432db578a80e0df5ba13` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.486591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] RBAC: Authorizing `identity:get_user(user_id=5091517659e7432db578a80e0df5ba13)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.488313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.490109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 184/369] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/users/5091517659e7432db578a80e0df5ba13 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.495942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.552841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.599725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821517b0-83b7-4ba6-9dd9-92888043c464 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.604660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abe082-d46d-4d9d-9a8f-217e0a05636d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.638145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.638853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.639131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.639469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.640255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:06.640626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.642567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.642679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.643555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.644021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.644459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.657983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 186/370] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.657983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.705633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.724832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.726342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.766182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b3086-2c88-42d2-9d3f-2a32267f86eb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.775657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.802547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.803418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.803857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.804284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.805148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:06.805601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.807167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.814229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 187/371] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.815598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.837136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 185/372] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:06.843685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:06.933792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf4c7e-a40d-414b-9d57-b04c29071d9c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:06.943050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:06.973445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:06.974439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:06.974982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:06.975476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:06.976408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] RBAC: Adding query filter params (role.id=248e2f3c1da941b8bc7e3551815d06a7, user.id=5091517659e7432db578a80e0df5ba13, scope.system=all) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:06.976933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:06.978593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:06.993558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 188/373] 10.222.0.15 () {56 vars in 1308 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/role_assignments?user.id=5091517659e7432db578a80e0df5ba13&scope.system=all&role.id=248e2f3c1da941b8bc7e3551815d06a7 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:06.994811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:07.000304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b7b79-d003-4102-bb17-606beb9be95e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.060981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.061605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.061941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.062207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.099697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:07.114795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204ade9-ba40-4b8c-8542-112538f80457 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.158865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.159885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.160381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.160848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.162362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.164053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.165953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 189/374] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:07.169985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.171856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.173030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:07.240341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:07.336339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 186/375] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:07.341793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:07.376408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38d5f5-46c4-4804-8ed4-a54837e7df65 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.408235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:07.445318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.445985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.446235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.446505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.448379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.449975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.451607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 190/376] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:07 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:07.458335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:07.493599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec1356-0eb0-4e26-91bb-4fe3c41fd96b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.538124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.538831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.539119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.539466 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] PATH_INFO: `/v3/projects/311e504f385e4c32bdb056e763a01299` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.540623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] RBAC: Authorizing `identity:get_project(project_id=311e504f385e4c32bdb056e763a01299)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.542962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.545128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 187/377] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:07 2022] GET /identity/v3/projects/311e504f385e4c32bdb056e763a01299 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:07.552175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:07.672080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bcbd2-db6c-40f5-a196-42a0f375a470 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.686367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:07.699336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcba7d-dd16-463d-852e-91b632185df9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.720628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:07.739326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.740231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.740706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.741153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.742071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:07.742561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.744381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.751929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 191/378] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:07 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=1cef455d27914904ba2d1176ced4233c => generated 260 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:07.758801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85178b-4a7b-45ae-af0e-73b72a27d9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.758801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85178b-4a7b-45ae-af0e-73b72a27d9c6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.758801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85178b-4a7b-45ae-af0e-73b72a27d9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.758801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85178b-4a7b-45ae-af0e-73b72a27d9c6 None admin] PATH_INFO: `/v3/groups/e2a36fb4dc3442af8e07ec3f4b76c614` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.776599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85178b-4a7b-45ae-af0e-73b72a27d9c6 None admin] RBAC: Authorizing `identity:get_group(group_id=e2a36fb4dc3442af8e07ec3f4b76c614)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.778206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85178b-4a7b-45ae-af0e-73b72a27d9c6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.779950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 188/379] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:07 2022] GET /identity/v3/groups/e2a36fb4dc3442af8e07ec3f4b76c614 => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:07.786222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:07.933923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365c4023-956a-4095-ac51-29a3be7cb739 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:07.970624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:07.971579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:07.972033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:07.972476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:07.973367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] RBAC: Adding query filter params (group.id=e2a36fb4dc3442af8e07ec3f4b76c614, role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=311e504f385e4c32bdb056e763a01299) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:07.973868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:07.975507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8196d7b4-19d0-4a8a-b990-e2f667f8ff5d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:07.982293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 192/380] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:07 2022] GET /identity/v3/role_assignments?group.id=e2a36fb4dc3442af8e07ec3f4b76c614&scope.project.id=311e504f385e4c32bdb056e763a01299&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:08.096984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32964b-61fc-4592-9be8-bc8f804e633f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.097565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32964b-61fc-4592-9be8-bc8f804e633f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.098030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32964b-61fc-4592-9be8-bc8f804e633f None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.099934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 189/381] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:08.106212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.106493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.106761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.127327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:08.129419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:08.154994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:08.216115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 193/382] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:08.257791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32964b-61fc-4592-9be8-bc8f804e633f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:08.381384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32964b-61fc-4592-9be8-bc8f804e633f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:08.433188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:08.433846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.434120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.434385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.449483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:08.454601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:08.455525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:08.456357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:08.460174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 190/383] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:08.465913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:08.621081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c847e4b-94ac-4ab0-95bd-42cdbfd45b12 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:08.668965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:08.669600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.669777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.669997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.670671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:08.670969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:08.676449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11451e25-8e9c-461b-973b-4000b7d01ade None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:08.690751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 194/384] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:08.696158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:08.771063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:08.845058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5ff23-c3e6-4575-be15-698a4ff7abe9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:08.851721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d0914-b0dd-48f5-b91c-45d950c7da9d None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.852006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d0914-b0dd-48f5-b91c-45d950c7da9d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.852284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d0914-b0dd-48f5-b91c-45d950c7da9d None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.855739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 195/385] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:08.859871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.860158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.860439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.879379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:08.880594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:08.891977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:08.892616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:08.892895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:08.893167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:08.894269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:08.895736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:08.897303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 191/386] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:08 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:08.903689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:08.911628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:09.006717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d49f0-1089-41b8-87fc-965ce62023a4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:09.021892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 196/387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:09.023368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:09.057141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:09.058109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.058463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.058746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.064031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:09.066065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:09.068139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 192/388] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:08 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:09.070162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.082240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.258114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016960f0-0fdb-4c37-9c3f-a0bf670b6d52 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:09.280104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7008d-4b34-496f-abf3-1ec2e0686dea None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:09.326705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e85f5a-ecdb-4415-bc3c-eb2a130ae30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:09.327406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85f5a-ecdb-4415-bc3c-eb2a130ae30d None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.327686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85f5a-ecdb-4415-bc3c-eb2a130ae30d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.327962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85f5a-ecdb-4415-bc3c-eb2a130ae30d None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.329362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:09.329990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.330261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.330531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.333207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e85f5a-ecdb-4415-bc3c-eb2a130ae30d None admin] RBAC: Authorizing `identity:create_grant(project_id=9804d64b9c1f43c59efe7e291a7b76e1, group_id=6927b755a9ad4f93b24d9112db6bf43e, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:09.335374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e85f5a-ecdb-4415-bc3c-eb2a130ae30d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:09.348809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:09.353927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:09.362020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 197/389] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:09.362801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.404549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:09.458180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 193/390] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 00:14:09 2022] PUT /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:09.464423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952d4e5-6a4d-4aa4-83a8-bb3bfdc533d4 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.464423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952d4e5-6a4d-4aa4-83a8-bb3bfdc533d4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.464423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952d4e5-6a4d-4aa4-83a8-bb3bfdc533d4 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.464423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 194/391] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:09.473664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.473981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.474229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.506711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:09.507982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:09.514684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ed455-0bfb-4e17-b60e-05903e7258e4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:09.576266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:09.589771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363f623-511c-47be-a380-03e92f473918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:09.590769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363f623-511c-47be-a380-03e92f473918 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.591212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363f623-511c-47be-a380-03e92f473918 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.591665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363f623-511c-47be-a380-03e92f473918 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.597347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363f623-511c-47be-a380-03e92f473918 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:09.597854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363f623-511c-47be-a380-03e92f473918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:09.599438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363f623-511c-47be-a380-03e92f473918 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:09.623355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 198/392] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:09.630795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363f623-511c-47be-a380-03e92f473918 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.673002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 195/393] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:09.695300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.734071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:09.780619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363f623-511c-47be-a380-03e92f473918 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:09.824365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:09.825609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.825992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.826369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.827819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:09.829496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:09.831821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 199/394] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:09 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:09.840302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.842772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f780481-a9b3-4951-b751-ab3184286b9d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:09.911907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:09.913302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:09.913782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:09.914183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:09.925984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:09.931428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:09.933023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:09.933968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:09.941678 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 196/395] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:09.943198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:09.987635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.052819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2213c9-feb1-4add-a78c-fbe5ca5ee5e3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.061963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e925b-ecaa-448b-8fa1-69f4c2ff3c04 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.089061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.089936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.090350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.090771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.091604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:10.092041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:10.093559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:10.115352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 197/396] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:10.118215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.148634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.149508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.149923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.150458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.152656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:10.155834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:10.162936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 200/397] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:09 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:10.164354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.178276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.306694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafcf56-9363-43e2-a91f-58d84dc7b577 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.321823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.337964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc6035-92c2-4047-9d0f-2d0c0eba7bce None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.384783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb3a8e-51f1-4dcb-81ce-694c802714be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.385824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb3a8e-51f1-4dcb-81ce-694c802714be None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.386238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb3a8e-51f1-4dcb-81ce-694c802714be None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.386678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb3a8e-51f1-4dcb-81ce-694c802714be None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.391832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb3a8e-51f1-4dcb-81ce-694c802714be None admin] RBAC: Authorizing `identity:create_grant(project_id=9804d64b9c1f43c59efe7e291a7b76e1, group_id=6927b755a9ad4f93b24d9112db6bf43e, role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:10.393445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb3a8e-51f1-4dcb-81ce-694c802714be None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:10.406250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.407186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.407648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.408053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.409407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:10.411009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:10.418342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 198/398] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:10 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:10.419824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.466514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 201/399] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 00:14:10 2022] PUT /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/1cef455d27914904ba2d1176ced4233c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:14:10.467885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27591-04d1-4ae2-aee5-b6b8fe80bbd9 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.468169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27591-04d1-4ae2-aee5-b6b8fe80bbd9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.468440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc27591-04d1-4ae2-aee5-b6b8fe80bbd9 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.474305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 202/400] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:10.475758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39924-c18b-46a6-80a6-7e17ffa0c4b6 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.476041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39924-c18b-46a6-80a6-7e17ffa0c4b6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.476312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad39924-c18b-46a6-80a6-7e17ffa0c4b6 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.477939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 203/401] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:10.479622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3a901-eb01-4d66-ab30-aa128dae7db7 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.479991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3a901-eb01-4d66-ab30-aa128dae7db7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.480455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3a901-eb01-4d66-ab30-aa128dae7db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.500308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-99d3a901-eb01-4d66-ab30-aa128dae7db7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:10.501531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-99d3a901-eb01-4d66-ab30-aa128dae7db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:10.551617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e69dcb-2f90-45d8-ab1a-c468959aefab None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.580686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.587065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 204/402] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:10.589230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.589565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.589888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.610105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:10.611973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:10.614566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.615220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.616445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.616737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.622557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:10.624477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:10.627002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 199/403] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:10 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:10.634281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.678767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.707864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 205/404] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:10.708824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.785474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352ad6f-a3f6-425d-96c9-d14ede940a82 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4de7b-eb74-41e6-9768-caf7f2db53ed None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:10.836244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.836801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.837321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.837602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.848829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.853540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:10.857487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:10.858268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:10.858602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:10.865569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:10.865720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:10.866729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:10.867664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:10.868270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] RBAC: Authorizing `identity:create_grant(project_id=ef07ebdea0ce49f1b264222c56073aa4, group_id=6927b755a9ad4f93b24d9112db6bf43e, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:10.868340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 200/405] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:10.868340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.868476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:10.957933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:10.964130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 206/406] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 00:14:10 2022] PUT /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:10.965763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:10.974842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ebe3cd-5992-4144-8204-102d63de70e5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.011109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.011781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.012056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.012323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.032696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:11.043002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:11.044020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:11.044809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:11.046132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 201/407] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:11.052022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.169847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2da7f-b721-4873-bd20-181a0a54b1b0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.171415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1390d0-57b4-4dd8-a986-b50467c2d8bd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.206874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.207579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.207853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.208122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.208796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:11.209094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.210587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.220103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 202/408] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:11.227161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.242120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:11.247394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.248018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.248295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.248565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.249231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:11.249679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.251516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.262263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 207/409] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:11.267919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.327764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:11.349863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305e9f-370b-49d3-b4fd-a71efac4f665 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.386837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3504b50-3425-4d89-9aad-d76abef61efc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.387891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.388526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.388802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.389073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] PATH_INFO: `/v3/groups/e2a36fb4dc3442af8e07ec3f4b76c614` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.390198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] RBAC: Authorizing `identity:get_group(group_id=e2a36fb4dc3442af8e07ec3f4b76c614)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.394146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.394146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 203/410] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:11 2022] GET /identity/v3/groups/e2a36fb4dc3442af8e07ec3f4b76c614 => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:11.400478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.458763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.459755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.460245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.460705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.465702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.467512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.472208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 208/411] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:11 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:11.488677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.489404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:11.514155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd46ab-2862-4d50-93fd-9025d2b29075 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.549753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.550377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.550693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.550967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] PATH_INFO: `/v3/projects/311e504f385e4c32bdb056e763a01299` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.552161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] RBAC: Authorizing `identity:get_project(project_id=311e504f385e4c32bdb056e763a01299)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.555541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.555541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 204/412] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:11 2022] GET /identity/v3/projects/311e504f385e4c32bdb056e763a01299 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:11.561493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.685422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d7eb2-f3d9-48fd-b91c-a65868f59961 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.724089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018adcb5-ac09-4d8c-97e1-ef8f26380a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.724771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018adcb5-ac09-4d8c-97e1-ef8f26380a62 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.725059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018adcb5-ac09-4d8c-97e1-ef8f26380a62 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.726179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018adcb5-ac09-4d8c-97e1-ef8f26380a62 None admin] PATH_INFO: `/v3/projects/311e504f385e4c32bdb056e763a01299/groups/e2a36fb4dc3442af8e07ec3f4b76c614/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.727715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018adcb5-ac09-4d8c-97e1-ef8f26380a62 None admin] RBAC: Authorizing `identity:create_grant(project_id=311e504f385e4c32bdb056e763a01299, group_id=e2a36fb4dc3442af8e07ec3f4b76c614, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.729495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018adcb5-ac09-4d8c-97e1-ef8f26380a62 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.742373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc5f52-99f3-4c5a-9a36-a3a21065915c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.771850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:11.780249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.780919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.781232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.781508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.782631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.784168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:11.786211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 209/413] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:11 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:11.791543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:11.808946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 205/414] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 00:14:11 2022] PUT /identity/v3/projects/311e504f385e4c32bdb056e763a01299/groups/e2a36fb4dc3442af8e07ec3f4b76c614/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:11.864915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87afac8-7c90-440a-b791-b08c7efc518a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.865267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87afac8-7c90-440a-b791-b08c7efc518a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.865549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87afac8-7c90-440a-b791-b08c7efc518a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.867659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 206/415] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:11.874935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.875273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.875553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.900445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:11.927798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c95b9-d4b6-4bf3-aaf6-7e6ded729f87 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:11.928037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:11.929208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:11.966245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:11.967004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:11.967315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:11.967738 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:11.970011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef07ebdea0ce49f1b264222c56073aa4, group_id=6927b755a9ad4f93b24d9112db6bf43e, role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:11.971518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:12.035980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 207/416] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:12.064077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 210/417] 10.222.0.15 () {58 vars in 1442 bytes} [Mon Aug 8 00:14:11 2022] PUT /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4/groups/6927b755a9ad4f93b24d9112db6bf43e/roles/1cef455d27914904ba2d1176ced4233c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:12.065133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:12.181026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d9504-e270-4e3c-b9ec-b12bc63f2faf None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:12.220072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:12.220721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:12.221003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:12.221278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:12.223268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:12.239617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:12.245381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:12.251794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:12.253644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 208/418] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:12.254509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:12.324637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:12.381437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50150c06-b523-408e-9531-c4a377fe5ed4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:12.424609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:12.425227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:12.425502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:12.425887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:12.426557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] RBAC: Adding query filter params (name=member) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:12.426854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:12.428262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:12.441317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 211/419] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:12.447263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:12.583063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2efcdde-44f8-46db-aab9-f215d93b5525 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:12.621340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:12.621956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:12.622233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:12.622506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:12.691653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:12.693963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:12.695582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:12.804425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 209/420] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:12.810489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:12.921208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61518db3-764f-4b02-b5c6-e648a0c38864 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:12.930725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:12.969235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abc017-fc42-4b87-9332-90c414e62a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:12.969888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abc017-fc42-4b87-9332-90c414e62a0b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:12.970169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abc017-fc42-4b87-9332-90c414e62a0b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:12.970472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abc017-fc42-4b87-9332-90c414e62a0b None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:12.971656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abc017-fc42-4b87-9332-90c414e62a0b None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:12.973190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abc017-fc42-4b87-9332-90c414e62a0b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:12.975059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 212/421] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:12 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:12.980602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.072204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714216f3-5127-4b60-bb45-236c280bb1ee None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.072628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714216f3-5127-4b60-bb45-236c280bb1ee None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.072986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714216f3-5127-4b60-bb45-236c280bb1ee None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.074811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 213/422] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:13.081092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.081376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.081652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.095641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:13.108288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaca78-17f5-42f6-a255-151bba03251e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:13.121603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:13.122824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:13.178127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:13.178676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.178958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.179246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.181645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:13.183611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:13.186894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 210/423] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:12 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:13.193508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.258367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 214/424] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:13.297286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.358152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd677c-41ba-4edf-9c85-384e443e42d8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:13.428016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:13.430356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.430881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.431386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.432701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:13.434350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:13.436791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da43e2-39eb-4ffe-a89d-e17e634b2ac1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:13.450173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 211/425] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:13 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 611 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:13.467281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb24c79-c8b7-40bf-9922-3ce4bb9fc07d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:13.476233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:13.485431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330bfd9-ccbc-4315-b669-c85b6aa31d68 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.485725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330bfd9-ccbc-4315-b669-c85b6aa31d68 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.491104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330bfd9-ccbc-4315-b669-c85b6aa31d68 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.491104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 212/426] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:13.503259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.503560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.503829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.531348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:13.532578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:13.542633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:13.543192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.543501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.543780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.566237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:13.571387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:13.572319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:13.573664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:13.575432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 215/427] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:13.582978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.587805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:13.670060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 213/428] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:13.697130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.729948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0e7af-a32a-4b21-8ba6-8a7ffbca8da7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:13.773555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:13.774708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.775210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.775698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.777106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:13.777818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:13.780102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:13.795326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 216/429] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:13.801964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.824447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e27efa-efa5-4c22-b56a-64fd65fbfc5d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:13.843412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:13.872193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:13.874435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.874921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.875425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:13.891824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:13.898201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:13.901363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 00:14:13.902166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:13.904487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 214/430] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:13.910759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:13.939251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:13.943975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07efd73b-3ad6-4214-9ed2-031f4bd3ba60 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:13.988085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:13.988731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:13.989249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:13.989538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.047362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e13f808-a8b4-4427-a2ab-7053d7dccfe3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.077996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.079522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.093000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.093900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.094182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.094457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.095344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] RBAC: Adding query filter params (name=member) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:14.096028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.097679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.108702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 215/431] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:14.116083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.116759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.215351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.217406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 217/432] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:14.223320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.242868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3997f4a-2b15-4899-ba6d-e508172cd63c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.295826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4263-2f89-4665-af4e-ed1c0e8467c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.296591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4263-2f89-4665-af4e-ed1c0e8467c0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.296902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4263-2f89-4665-af4e-ed1c0e8467c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.297180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4263-2f89-4665-af4e-ed1c0e8467c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.349274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc03cd-fd81-4bb9-831c-ca0835f8435c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.388072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.402589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4263-2f89-4665-af4e-ed1c0e8467c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.404004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.404511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4263-2f89-4665-af4e-ed1c0e8467c0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.404710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.404982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.405262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] PATH_INFO: `/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.406704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] RBAC: Authorizing `identity:get_project(project_id=9804d64b9c1f43c59efe7e291a7b76e1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.409216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.411180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 218/433] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/projects/9804d64b9c1f43c59efe7e291a7b76e1 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:14.418218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.496379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.518754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 216/434] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:14.520044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f053364-6578-4eb1-9cd9-d0336a2d3e89 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.520330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f053364-6578-4eb1-9cd9-d0336a2d3e89 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.520607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f053364-6578-4eb1-9cd9-d0336a2d3e89 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.522094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 217/435] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:14.528177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f053364-6578-4eb1-9cd9-d0336a2d3e89 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.532270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f240fb-57ac-44d4-9c35-c9d7effd84ba None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.569478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d00d0-b6e7-46be-9285-b5b32175c8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.570106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d00d0-b6e7-46be-9285-b5b32175c8a0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.570383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d00d0-b6e7-46be-9285-b5b32175c8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.570655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d00d0-b6e7-46be-9285-b5b32175c8a0 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.571784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d00d0-b6e7-46be-9285-b5b32175c8a0 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.573823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d00d0-b6e7-46be-9285-b5b32175c8a0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.575407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 219/436] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:14.576578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.576857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.577125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.595135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:14.596330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:14.656373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f053364-6578-4eb1-9cd9-d0336a2d3e89 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.660373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.668753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 220/437] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:14.669580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.706506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.707453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.707732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.708003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.709114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.710541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.712173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 218/438] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:14.716151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.761368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.778048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153208c2-4199-4b5c-b872-e7a08c8bec2d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.811702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41442cd-5782-4406-8426-5758bc90ccf6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.813348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.813954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.814226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.814490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.815198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:14.815796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.817214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee1b2e-61a0-4903-867e-63c597a21a11 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:14.823704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 221/439] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=9804d64b9c1f43c59efe7e291a7b76e1&role.id=1cef455d27914904ba2d1176ced4233c => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:14.824863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735601cf-0902-45e2-bd7e-6216f3b37d8a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.825137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735601cf-0902-45e2-bd7e-6216f3b37d8a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.825416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735601cf-0902-45e2-bd7e-6216f3b37d8a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.826592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 222/440] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:14.827375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735601cf-0902-45e2-bd7e-6216f3b37d8a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.857721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b86a25-ea85-4c9c-b06c-1314099b4a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.858374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b86a25-ea85-4c9c-b06c-1314099b4a97 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.858650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b86a25-ea85-4c9c-b06c-1314099b4a97 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.858927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b86a25-ea85-4c9c-b06c-1314099b4a97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.871383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b86a25-ea85-4c9c-b06c-1314099b4a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-17b86a25-ea85-4c9c-b06c-1314099b4a97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:14.876281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:14.877250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:14.878063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 219/441] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:14.879007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.879307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.879581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.899125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:14.900383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:14.903538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:14.955498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735601cf-0902-45e2-bd7e-6216f3b37d8a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:14.975064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 220/442] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:14.976312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:14.994107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:14.995435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:14.995919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:14.996385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:14.997535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:14.998974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.000880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 223/443] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:15.001640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.027471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.110939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70586ad4-bf6e-414a-bc73-9c31e1e9a882 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.127436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28539521-9c6a-4b01-89cd-daa731ca6d35 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.155151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.155808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.156082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.156352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.157020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:15.157327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.157866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.158706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.169489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 221/444] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:15.170719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.170817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.171652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.172119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.172393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.184892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:15.190330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:15.191550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:15.192386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 224/445] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:15.193050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.276875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.280354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597c947-009d-4615-9f99-2db40de26bac None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.299617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f6eb25-55a8-477e-b2a6-d8696970bfa0 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.313786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.314386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.314665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.314941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.315967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=094f7bfb155b40fea515d8a0935ab3db, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:15.316276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.317740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.323570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 222/446] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=094f7bfb155b40fea515d8a0935ab3db => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:15.324347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.333252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.334051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.334166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.334417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.400242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.415794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.418230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.443917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef04944-cb22-4429-aee9-37edbf1f3bf2 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.487313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.487927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.488207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.488483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.489175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:15.489477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.490862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.500121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 223/447] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:15.505864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.533370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 225/448] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:15.533864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.539902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.623531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75414445-2fd3-403a-a9ab-cf78f199693f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.642494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.663845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6436dd7-7e93-468e-b222-0c252b2bf5a6 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.665086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.665697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.665978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.666256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.703535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.704157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.704430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.704699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] PATH_INFO: `/v3/projects/311e504f385e4c32bdb056e763a01299` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.705799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] RBAC: Authorizing `identity:get_project(project_id=311e504f385e4c32bdb056e763a01299)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.707324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.708851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 226/449] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/projects/311e504f385e4c32bdb056e763a01299 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:15.714769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.739616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.741083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.764512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.811301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbaf52-2e8d-45c6-b4f1-ba6593f8ce8f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.839546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 224/450] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:14:15.844235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.844913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.845080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.845173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.845449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] PATH_INFO: `/v3/groups/e2a36fb4dc3442af8e07ec3f4b76c614` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.846551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] RBAC: Authorizing `identity:get_group(group_id=e2a36fb4dc3442af8e07ec3f4b76c614)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.848017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.849556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 227/451] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/groups/e2a36fb4dc3442af8e07ec3f4b76c614 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:15.856204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.880594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.945925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749ff0-9c0a-4ee8-8e3b-1e920b351428 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.956811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b27a4f-69d3-4fb0-b0a7-6a99672f2eda None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:15.968769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:15.979477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.980163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.980439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.980711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] PATH_INFO: `/v3/projects/ef07ebdea0ce49f1b264222c56073aa4` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.981841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] RBAC: Authorizing `identity:get_project(project_id=ef07ebdea0ce49f1b264222c56073aa4)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.983290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.984936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 225/452] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/projects/ef07ebdea0ce49f1b264222c56073aa4 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:15.989920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:15.990401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:15.990506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:15.990776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:15.991044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:15.991785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] RBAC: Adding query filter params (group.id=e2a36fb4dc3442af8e07ec3f4b76c614, role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=311e504f385e4c32bdb056e763a01299) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:15.992088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:15.993495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:15.999989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 228/453] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/role_assignments?group.id=e2a36fb4dc3442af8e07ec3f4b76c614&scope.project.id=311e504f385e4c32bdb056e763a01299&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:16.096919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21a357-1ee7-4fe5-a80d-2b73d8046651 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:16.132106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a413a-37a3-4689-b293-54b28eb7a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:16.132744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a413a-37a3-4689-b293-54b28eb7a78c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:16.133022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a413a-37a3-4689-b293-54b28eb7a78c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:16.133292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a413a-37a3-4689-b293-54b28eb7a78c None admin] PATH_INFO: `/v3/groups/6927b755a9ad4f93b24d9112db6bf43e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:16.134452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a413a-37a3-4689-b293-54b28eb7a78c None admin] RBAC: Authorizing `identity:get_group(group_id=6927b755a9ad4f93b24d9112db6bf43e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:16.136867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a413a-37a3-4689-b293-54b28eb7a78c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:16.138485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 226/454] 10.222.0.15 () {56 vars in 1180 bytes} [Mon Aug 8 00:14:15 2022] GET /identity/v3/groups/6927b755a9ad4f93b24d9112db6bf43e => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:16.144415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:16.232234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:16.246776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b937528-fb25-4131-958f-642166e0c9fd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:16.279675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:16.280272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:16.280548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:16.280816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:16.281530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] RBAC: Adding query filter params (group.id=6927b755a9ad4f93b24d9112db6bf43e, role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=ef07ebdea0ce49f1b264222c56073aa4) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:16.281835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:16.283273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e8f3d-b1f8-492b-86dc-8bccb4b35049 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:16.288791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 229/455] 10.222.0.15 () {56 vars in 1376 bytes} [Mon Aug 8 00:14:16 2022] GET /identity/v3/role_assignments?group.id=6927b755a9ad4f93b24d9112db6bf43e&scope.project.id=ef07ebdea0ce49f1b264222c56073aa4&role.id=1cef455d27914904ba2d1176ced4233c => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:18.766216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750f6d9-8889-45b0-b318-f85a0806386c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:18.767051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750f6d9-8889-45b0-b318-f85a0806386c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:18.767484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750f6d9-8889-45b0-b318-f85a0806386c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:18.769281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 227/456] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:18.775219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea1914-95af-4344-8583-ae477fff046d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:18.776085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea1914-95af-4344-8583-ae477fff046d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:18.776507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea1914-95af-4344-8583-ae477fff046d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:18.804201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8bea1914-95af-4344-8583-ae477fff046d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:18.805610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8bea1914-95af-4344-8583-ae477fff046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:18.904087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 230/457] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:18.939753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750f6d9-8889-45b0-b318-f85a0806386c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.018972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92035f7f-2d51-447a-84b4-d39946829de8 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.019509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92035f7f-2d51-447a-84b4-d39946829de8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.019922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92035f7f-2d51-447a-84b4-d39946829de8 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.021499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.022874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 231/458] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:19.028680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726a39f-d6ad-43d4-8978-9158fdca0305 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.036730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726a39f-d6ad-43d4-8978-9158fdca0305 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.036730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726a39f-d6ad-43d4-8978-9158fdca0305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.074618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750f6d9-8889-45b0-b318-f85a0806386c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.080308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8726a39f-d6ad-43d4-8978-9158fdca0305 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:19.081511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8726a39f-d6ad-43d4-8978-9158fdca0305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:19.121584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.122451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.122899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.123419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.134107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.136059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-349f2ede-9aa7-4f23-baa3-cd41d688b3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:19.151731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:19.152837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:19.153848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:19.153848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:19.153848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:19.155590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 228/459] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:19.157056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4233-cf37-47b9-955c-c5724b1b4549 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.157538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4233-cf37-47b9-955c-c5724b1b4549 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.158005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4233-cf37-47b9-955c-c5724b1b4549 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.159464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 229/460] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:19.161123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4233-cf37-47b9-955c-c5724b1b4549 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.167365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 232/461] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:19.168746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.169030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.169298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.188320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:19.189912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:19.193394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.270948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 233/462] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:19.271645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4233-cf37-47b9-955c-c5724b1b4549 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.272183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.285983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.308683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.309670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.310145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.310611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.311539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:19.312039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.313622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.327122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 230/463] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:19.328347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.395280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe37619-b56a-447d-91c0-2129863961ca None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.435253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.435253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.435253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.435253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.435649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37a0ed-2e64-43ed-b2ee-aa811301005d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.444681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.448267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.450621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:19.471092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:19.472188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:19.473033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:19.473033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:19.473033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:19.475726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 234/464] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:19.477257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.480200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.480968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.481249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.481516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.495127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.497280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:19.518343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:19.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:19.520627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:19.520627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:19.520627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:19.523308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 231/465] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:19.525301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.560167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.610627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414fe3b7-e7d5-4f27-9a47-cca0d1e394e7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.661131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.661367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231072a-9dd6-4992-aa9a-b748bd09ad89 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.662100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.662521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.663182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.664971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.674400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.699557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.700171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.700517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.700962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.701650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:19.701953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.703346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.715536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 232/466] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:19.716241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.789552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 235/467] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:14:19 2022] POST /identity/v3/users => generated 246 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:19.790401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.802815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.821556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6cfccb-0806-4e10-b6ec-ca92e2cccc93 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:19.860011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:19.861246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.861735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.862154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.862994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:19.863328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:19.864745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f737cf-51f7-495a-92c4-003df37706f9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:19.867902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:19.879145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 233/468] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:19.880546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d4841-e90c-47f2-b7bd-ed381b238462 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:19.880834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d4841-e90c-47f2-b7bd-ed381b238462 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:19.881113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d4841-e90c-47f2-b7bd-ed381b238462 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:19.887643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 234/469] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:19.888547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d4841-e90c-47f2-b7bd-ed381b238462 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:19.948837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f321a-dfbc-4a96-af7b-5d95ba41b39e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:20.002248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea271a-c737-4c39-a844-25f80e465f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:20.002781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea271a-c737-4c39-a844-25f80e465f11 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.003057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea271a-c737-4c39-a844-25f80e465f11 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.004095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea271a-c737-4c39-a844-25f80e465f11 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.004095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea271a-c737-4c39-a844-25f80e465f11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:20.006223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ea271a-c737-4c39-a844-25f80e465f11 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:20.021438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d4841-e90c-47f2-b7bd-ed381b238462 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:20.060291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:20.060897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.061171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.061438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.062073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:20.063690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:20.084038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 236/470] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:14:19 2022] POST /identity/v3/users => generated 248 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:20.085399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.085684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.085954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.108697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:20.110117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:20.169824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:20.178222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 235/471] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:14:19 2022] POST /identity/v3/users => generated 248 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:20.199743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 237/472] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:20.224859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:20.363385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99d2ee-f430-477d-aed0-3f557ee41b95 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:20.401540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:20.402586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.403036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.403481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.416172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:20.418243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-fe16a46e-6771-46dc-8a4b-9a87243e4b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:20.434302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:20.435573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:20.436464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:20.436464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:20.436464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:20.436464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 236/473] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:20.443014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:20.453462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf93f-fdbb-416a-a815-bb1e762db550 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.453750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf93f-fdbb-416a-a815-bb1e762db550 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.454018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bf93f-fdbb-416a-a815-bb1e762db550 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.455725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 237/474] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:20.460969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.461247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.461614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.483270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:20.484479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:20.485285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:20.549594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:20.575342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cf3f5-8229-4af4-87bd-75eaf24de086 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:20.580764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 238/475] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:20.620629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:20.623276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:20.624789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.625072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.625338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.625988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:20.626283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:20.627686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:20.650009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 238/476] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:20.656033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:20.736570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897a361-4bf6-4f8a-a719-1d10150fcf8b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:20.771529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:20.772144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.772424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.772703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.773362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:20.774877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:20.779042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:20.788330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae3823-79b9-4ecb-b7fd-567af161bf49 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:20.824892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:20.825517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:20.825826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:20.826156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:20.826813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:20.828286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:20.837769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.sql.core [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 00:14:20.837769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 00:14:20.837769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [parameters: {'id': '43abce1c5056424c8347ede95b381dca', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 00:14:20.837769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91988) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application conn.query(q) Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result.read() Aug 08 00:14:20.860186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 00:14:20.861145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application next(self.gen) Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application yield resource Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application next(self.gen) Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise self.value Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application session.commit() Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 00:14:20.862036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.session.flush() Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application compat.raise_( Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 00:14:20.862886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 00:14:20.863779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application conn.query(q) Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result.read() Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 00:14:20.864588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [parameters: {'id': '43abce1c5056424c8347ede95b381dca', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:20.865375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 00:14:20.866176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:20.871620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 239/477] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 00:14:20 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:20.871620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:20.925817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 239/478] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:14:20 2022] POST /identity/v3/users => generated 249 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:14:20.937184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:20.987762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40df1f-3544-4054-8826-b3508e174ee5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:21.024522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5de979-a93d-4233-96c7-2732535d70e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:21.025427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5de979-a93d-4233-96c7-2732535d70e3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:21.026054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5de979-a93d-4233-96c7-2732535d70e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:21.026523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5de979-a93d-4233-96c7-2732535d70e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:21.038451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5de979-a93d-4233-96c7-2732535d70e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-2c5de979-a93d-4233-96c7-2732535d70e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:21.044339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:21.045453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 00:14:21.046416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:21.052968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 240/479] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:21.059610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:21.204710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1520a68-dfb2-4ed7-b5b3-0c5c7ed047fe None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:21.242385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:21.243001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:21.243376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:21.243653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:21.244326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:21.245134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:21.246988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837ebc4-b163-459e-a3fe-33c04d16f496 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:21.258008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 240/480] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:22.301119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a357-f289-4796-a456-9051af691a05 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.302508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a357-f289-4796-a456-9051af691a05 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.302929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9a357-f289-4796-a456-9051af691a05 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.304685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 241/481] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:22.310921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.311396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.311486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.340404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:22.341608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:22.353329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:22.427702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 241/482] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:22.470906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9a357-f289-4796-a456-9051af691a05 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:22.621786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9a357-f289-4796-a456-9051af691a05 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:22.629377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:22.675711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d167fc64-dd0d-417b-a84e-3d366e07b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:22.676399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167fc64-dd0d-417b-a84e-3d366e07b463 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.676689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167fc64-dd0d-417b-a84e-3d366e07b463 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.677148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d167fc64-dd0d-417b-a84e-3d366e07b463 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.693927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d167fc64-dd0d-417b-a84e-3d366e07b463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d167fc64-dd0d-417b-a84e-3d366e07b463 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:22.699275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:22.700240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:22.701031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 242/483] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:22.706748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:22.733970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9198e-5e39-4ca2-b034-1dc84e7f1cef None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.734591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9198e-5e39-4ca2-b034-1dc84e7f1cef None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.735277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a9198e-5e39-4ca2-b034-1dc84e7f1cef None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.737137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 243/484] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:22.742058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac387-3f17-4bc9-b25c-e91c6e6e817f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.742580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac387-3f17-4bc9-b25c-e91c6e6e817f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.743059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dac387-3f17-4bc9-b25c-e91c6e6e817f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.777996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b5dac387-3f17-4bc9-b25c-e91c6e6e817f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:22.779679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b5dac387-3f17-4bc9-b25c-e91c6e6e817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:22.821737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92faea9c-ecf3-451a-9350-fcb75c5317d6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:22.877657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:22.878188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.878466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.878733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.879417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:22.879861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:22.881620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc5300-3481-43e5-81c1-2cba0b5fffc5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:22.892601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 242/485] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:22.893533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 244/486] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:22.893886 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbbe0f-5fb5-486b-b9e6-96fd541923a7 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.894171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbbe0f-5fb5-486b-b9e6-96fd541923a7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.894443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbbe0f-5fb5-486b-b9e6-96fd541923a7 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.899325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 243/487] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:22.901809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:22.902084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:22.902351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:22.904457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbbe0f-5fb5-486b-b9e6-96fd541923a7 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:22.922582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:22.937071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:22.938897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:23.026100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 245/488] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:23.027065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.044473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbbe0f-5fb5-486b-b9e6-96fd541923a7 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.078046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.079626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.080362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.080704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.081420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.092955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.094558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:23.108835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:23.109748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:23.110620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:23.110620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:23.110620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:23.110620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 244/489] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:23.111495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.143441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a754f-19ae-4165-ab9c-8908b245607d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.172345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.181896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.183566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.188157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.188157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.195639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:23.200584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:23.201489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:23.202265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:23.209495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 246/490] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:23.210561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.223856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463dcd0-2918-4b76-94f0-f22e40379a77 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.259424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.260206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.260907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.261281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.271797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:23.277087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:23.277990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:23.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:23.283523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 245/491] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:23.284463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.319617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.328704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fbcbf-0d94-4a43-9c19-33dc1c9e4bfd None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.369049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.370680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.371210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.371877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.372814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:23.373305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.375032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.392259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38473b46-772d-4a59-bb23-fa3ea83bc524 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.393046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 247/492] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:23.394176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.407417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.426712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.434597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.434597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.434597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.434597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:23.434597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.434597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.445187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 246/493] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:23.445187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.516661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66068feb-2c0b-4209-8285-6e41c46e3a50 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.556545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.557864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.558532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.559192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.560168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:23.560680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.562347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.566684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602d52e-0440-4287-a871-0f0b4c52901a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.572379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 248/494] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:23.573514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.591593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.610660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.611347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.611751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.612029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] PATH_INFO: `/v3/users/nova` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.620331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.622189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:23.641669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:23.642731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:23.643660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:23.647831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 247/495] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:23.650084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.700046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdb92f-56c1-46ca-8754-f12d51bece23 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.702528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.739821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.740293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.740772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.752906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.754738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:23.773400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:23.774484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:23.775355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:23.775355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:23.775355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:23.777997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 249/496] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:23.780007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.783667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddce6f-c3a7-4e0d-8d90-0848daf5afac None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.820529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.821160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.821435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.821700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.832660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.834140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.848236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-764ce492-adff-4973-a45c-99b3f4148dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:23.849257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:23.850142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:23.851006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:23.851006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:23.851006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:23.851006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 248/497] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:23.851538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96f287-f8d2-4551-ad16-bef30e2a4ac8 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.851802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96f287-f8d2-4551-ad16-bef30e2a4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.852069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96f287-f8d2-4551-ad16-bef30e2a4ac8 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.853280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 249/498] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:23.855024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bcd71-fb5f-4694-a4ad-85eb5af1576a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.855340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bcd71-fb5f-4694-a4ad-85eb5af1576a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.855658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bcd71-fb5f-4694-a4ad-85eb5af1576a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.872429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 250/499] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:23.872429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bcd71-fb5f-4694-a4ad-85eb5af1576a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:23.899890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c2f25-0923-4c1f-b23e-02a4da3b1c7f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.936991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:23.938014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:23.938292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:23.938563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:23.939323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:23.939621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:23.941340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:23.954028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:23.966694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bcd71-fb5f-4694-a4ad-85eb5af1576a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:23.978726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 250/500] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:23.979824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.009949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.011087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.011474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.011750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.012385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:24.012682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.014048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e653a-ae6b-466b-95d0-91a4bd29d324 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.025616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 251/501] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:24.027709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94895a8-1184-4e62-ae67-e8a1c6d88805 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.027995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94895a8-1184-4e62-ae67-e8a1c6d88805 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.028280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94895a8-1184-4e62-ae67-e8a1c6d88805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.045924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e94895a8-1184-4e62-ae67-e8a1c6d88805 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:24.047140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e94895a8-1184-4e62-ae67-e8a1c6d88805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:24.082231 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95036c7-176e-4eb9-9912-b8ab0289cd31 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.102114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.115308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 252/502] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:24.116560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.116845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.117120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.126682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.127476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.127856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.128225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.128965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:24.129367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.130915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.136170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:24.137479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:24.146613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 251/503] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:24.148498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.206733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 253/504] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:24.207627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.220422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.272905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4b560-4c6e-4c26-a086-e9ce1580139b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.311387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.311999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.312283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.312677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.316674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59141804-3fe9-40eb-9e1c-52c8984a3c12 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.358396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.359506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.360005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.360771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.360881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.368874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.371204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.382011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:24.383110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:24.384021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:14:24.384949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:24.386097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.386173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 254/505] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:24.386173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.453794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.483467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99409ce-83c8-4bee-96b3-c5bf8f7b9976 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.492746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 252/506] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:24.493593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.522855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.523508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.523788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.524062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.534262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:24.539399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:24.540460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:24.541477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 255/507] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:24.542304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.571655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.593931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a708-0c41-4b12-afa8-6d2fc88aaf73 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.629831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.630455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.630733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.631002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.638007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.639500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.646869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921400bf-b3c1-4746-be90-4c8fd2115ca7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:24.652115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:24.653245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:24.654151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 253/508] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:24.654717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.680877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.687212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.691525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.691525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.691525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.699105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.700650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:24.714900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:24.715970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:24.716988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:24.716988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:24.716988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:24.716988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 256/509] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:24.716988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.760460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf555e47-efc5-43ec-9a5d-6010121131b6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.793165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.793766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.794047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.794315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.795030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:24.795361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.796804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.805311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.814325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 254/510] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:24.815142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.828891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d2da7-5da8-49a9-aba7-29cfeaecad11 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.857209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.857836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.858117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.858387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.869221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.870939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:24.884598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:24.886218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:24.887164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:24.887164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:24.887164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:24.887164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 257/511] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:24.887164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.914551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb376ebe-1492-4165-80fc-bd20b2e31376 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:24.922429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:24.949862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:24.950485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:24.950767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:24.951036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:24.951722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:24.952026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:24.953410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:24.961211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 255/512] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:24.962079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:24.991062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cecf8c-a178-419e-b64b-19caae991a87 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.018936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.025763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.026358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.026631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.026902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.027651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:25.027954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.029379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.059371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd887b1d-1bda-4d6b-a8b8-fffa92341219 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.064748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 258/513] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:25.065400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.093307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.093950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.094227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.094496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.095119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:25.095441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.096843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.108088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 256/514] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:25.108883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.159284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.167662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4a519-2c5a-4a08-ae54-c7674a8f9775 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.205994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.206207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02d94c-819b-46ed-9c49-b9d9ce8598da None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.206603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.206884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.207160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.223172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.239608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.240201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.240478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.240742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.241364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:25.241657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.243005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.254893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 257/515] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:25.255701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.274319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.275785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.358028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259e63a-d465-4a6d-9e89-6815689cfdb5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.365131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.374823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 259/516] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:25.375661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.393397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.394017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.394295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.394564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.405701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.407151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:25.421279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:25.422203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:25.423052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:25.423052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:25.423052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:25.423052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 258/517] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:25.423311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.457343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.476099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c003-9a4c-4a64-8242-9e3ecbc4b629 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.509718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.510357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.510740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.511011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.525628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b7c203-3e84-4726-bbf0-384ef00bf76a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.560377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.560968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.561247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.561517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.562144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.563601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.568391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.577686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.579143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.673400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 259/518] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:14:25 2022] POST /identity/v3/users => generated 247 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:25.673400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.684503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 260/519] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:25.685388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.779769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.791145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:25.796378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f0d0bf-25d7-4f26-ad58-1d46b83e76fb None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.801412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e84b6-b7d0-4204-8cfa-dbc8f1caf022 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.834952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.835863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.835863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.836131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.843188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:25.843382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.843617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:25.844472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:25.844472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:25.844816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.855535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:25.856573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:25.857395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:25.858636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:25.859337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:25.859411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 260/520] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:25.859411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:25.877015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:25.877967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:25.878823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:25.878823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:25.878823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:25.882659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 261/521] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:25.884772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:25.971844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d891ad-085b-499b-9219-190b1a9f71c0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:25.995404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235dfcd-49e4-46f4-9b85-c3fe988b5003 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.006635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.007254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.007534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.007803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.008463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.008763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.010179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.026816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 261/522] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.026816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.036002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.036897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.037586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.037870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.038328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.050220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.051699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.059288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:26.068108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:26.069052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:26.069929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:26.069929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:26.069929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:26.074705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 262/523] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:26.075621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.137503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e812e3-c407-4b33-8f88-52eff0ca6c2f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.173812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.174856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.175307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.175797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.176611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.177067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.178257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864889d8-9913-42c5-882e-bb127a6e25a8 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.178797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.197683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 262/524] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.198345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.221517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.222125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.222402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.222673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.223340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.223642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.225012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.237362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 263/525] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.238181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.273378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.296831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.310943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b006d9-b432-44d3-86d4-7060dd0e7ee7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.343994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5df2f-8726-4555-a998-0969126c8733 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.348977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.349604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.349881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.350149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.358044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.359541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:26.372329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:26.373290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 263/526] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:26.374261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.385687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.386272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.386568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.386821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.387487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.387781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.389142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.405843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 264/527] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.405843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.482304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54009ead-9145-46fe-8621-ecaadc56de36 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.509107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.511446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b01237-4c99-478d-bec7-90b72b0372d6 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.518545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.519136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.519441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.525435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.525435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=c1031699d7da4a76965035faa9305dbd) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.525435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.525435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.528912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 264/528] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/role_assignments?user.id=c1031699d7da4a76965035faa9305dbd&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.536857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.545595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.565600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.566656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.567415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.567694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.576946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.578796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:26.592763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:26.593687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:26.594531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 265/529] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:26.595486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.667662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2c748-5ecb-41c1-b091-94d72f475e40 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.713625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.714319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.714594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.714861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.715629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.715928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.717360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.732030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2238a-3424-4fbf-9002-02b492f6a6aa None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.748023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 265/530] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.748023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.777015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.778123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.778624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.779189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.786878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.788650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.801790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:26.803481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:26.804436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:26.805275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:26.808392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 266/531] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:26.809379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.851736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:26.876329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebffb29-c78e-4336-af5d-d0f0e08b23f4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.920202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37863877-2fc8-4e14-955a-0b8b56662dc8 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:26.920622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.921226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.921503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.921988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:26.922673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:26.923607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:26.925055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:26.942897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 266/532] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:26.943822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:26.965678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:26.966287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:26.966562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:26.966830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.052433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d61f51-eb1e-4797-b9cf-458058829d6d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:27.061329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.062171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:27.064044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.089626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:27.090758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.091300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.091676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.092431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:27.092801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.094316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.112836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 267/533] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:27.112836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:27.136095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:27.196640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 267/534] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:27.198372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:27.221816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2686b-ed94-4c82-9ce2-deb65aedd32f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:27.266951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:27.268410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.268862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.269327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.270214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=2e66698136f4429b8df67e5050a8da2f) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:27.270880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.273014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.281461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 268/535] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:27 2022] GET /identity/v3/role_assignments?user.id=2e66698136f4429b8df67e5050a8da2f&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:27.282633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:27.321803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:27.365384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0776ad6-7eed-4db4-baa8-b85c24916c4e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:27.428443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8053ac8-54e3-4b1d-a69e-99c982359b06 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:27.434158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:27.434793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.435086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.435361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.436092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=de6c52a7640e47c7be2253e32aa5a173) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:27.436392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.437843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.447781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:27.452568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 268/536] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:27 2022] GET /identity/v3/role_assignments?user.id=de6c52a7640e47c7be2253e32aa5a173&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:27.478595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:27.479533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.479731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.479984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.492905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.494973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:27.511211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:27.512207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:27.513520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:27.513520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:27.513520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:27.513520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 269/537] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:27.519665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:27.645562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd3edd-6c57-4488-8c1e-27466c8a2063 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:27.692639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:27.693317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.693598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.693869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.694510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:27.694807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.696202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022bb05-9986-452e-9e09-68cff6f5ef98 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.710564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 269/538] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:27.716398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:27.837613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50e0c8-05f9-4e43-a4ba-b2cf260595cc None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.837935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50e0c8-05f9-4e43-a4ba-b2cf260595cc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.838701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50e0c8-05f9-4e43-a4ba-b2cf260595cc None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.840693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 270/539] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:27.850699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.851114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.851293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:27.851448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.868052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afe8a7-55d6-47bc-8585-072aa3438fb2 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:27.889299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:27.892249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:27.932060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:27.933180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:27.933636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:27.934139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:27.946408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:27.949481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:27.967542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:27.968424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:27.969283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:27.971763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 270/540] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:27.977526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:27.995600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:28.017523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 271/541] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:28.041964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:28.172862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332fa98-433e-4c7c-8625-8e7fb9a105e0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:28.233909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b36e4-d064-40f3-a57c-187c3e720546 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:28.266106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:28.266722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.266996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.267285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.267978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:28.268278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:28.269703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:28.285497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:28.286132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.286406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.286669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.301251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 271/542] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:28.305576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:28.307892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:28.312689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:28.313580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:28.314392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:28.318322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:28.331327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 272/543] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:28.335196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:28.442080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee5a43-c6f4-4859-a28e-de23628b7580 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:28.452261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:28.486831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:28.487447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.487733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.488002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.488727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=6c8a8736fc89485d87f14e06e85e67c1) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:28.489031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:28.490460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:28.502305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 272/544] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:28 2022] GET /identity/v3/role_assignments?user.id=6c8a8736fc89485d87f14e06e85e67c1&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:28.526650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7b703-d164-4597-947f-94d51ebec17f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:28.579838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:28.580917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.581332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.581730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.582419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:28.582727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:28.584140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b0807-c75a-4601-a23b-6c545415d5df None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:28.597918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 273/545] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:28.603570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:28.708400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b38bd-e442-4e6b-aa47-6329a15fa6a8 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.708722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b38bd-e442-4e6b-aa47-6329a15fa6a8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.708945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b38bd-e442-4e6b-aa47-6329a15fa6a8 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.710852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 274/546] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:28.716190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.716486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.716725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.736479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:28.746554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:28.747840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:28.791318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb30f0-7e73-4694-a2a3-249ed95ebddb None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:28.865131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:28.865744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:28.866024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:28.866296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:28.870437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 275/547] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:28.889214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:28.890673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:28.914973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:28.916050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:28.917524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:28.917524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:28.917524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:28.917524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 273/548] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:28.920978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:28.923706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.000644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:29.100195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caacf0-aae0-43f4-9819-84b4a35a5e6f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.109071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecabdc-9a81-48f7-8560-7b090eca5a0f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.133569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:29.138131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.138887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.139173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.139471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.149757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:29.154483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:29.155613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:29.156443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 276/549] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:29.161586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.171117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26679547-375b-4125-877a-14deacf11c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.171767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26679547-375b-4125-877a-14deacf11c76 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.172042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26679547-375b-4125-877a-14deacf11c76 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.172310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26679547-375b-4125-877a-14deacf11c76 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.175651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26679547-375b-4125-877a-14deacf11c76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:29.175963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26679547-375b-4125-877a-14deacf11c76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:29.177403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26679547-375b-4125-877a-14deacf11c76 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:29.202714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 274/550] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:29.208261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26679547-375b-4125-877a-14deacf11c76 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.352483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157506ec-c073-445b-97fa-c2e9269c7643 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.359558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:29.406148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26679547-375b-4125-877a-14deacf11c76 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.420164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.421128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.421601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.422065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.422926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:29.423658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:29.425236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:29.435196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 277/551] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:29.441018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.472261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.472891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.473165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.473429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.481488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:29.482963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:29.497159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:29.498103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:29.499032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:29.502939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 275/552] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:29.504115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.551092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:29.613788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4acfa0-cdcd-4f5e-92b8-a61e8ae11444 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.661933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.662698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.662980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.663273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.664005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:29.664310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:29.665735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:29.694446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 276/553] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 00:14:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:29.695719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.749182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c2d7d-de3d-40be-add2-bc0e170a1b48 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.804655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:29.810004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcee61-d580-4391-9425-2acccf6f930d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:29.822440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.823110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.823432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.823701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.834514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:29.836162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:29.849007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fc78d-29dd-42ca-a50a-5feb5f01c163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:29.849612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fc78d-29dd-42ca-a50a-5feb5f01c163 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.849893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fc78d-29dd-42ca-a50a-5feb5f01c163 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.850166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fc78d-29dd-42ca-a50a-5feb5f01c163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-c61e2d26-1db7-4e64-89ee-3d688d7893bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:29.850858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:29.851766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:29.852654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:29.852654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:29.852654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:29.857154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 278/554] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:29.858392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67241884-da4c-4e34-86f3-721d8bc4b4be None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.858671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67241884-da4c-4e34-86f3-721d8bc4b4be None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.858941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67241884-da4c-4e34-86f3-721d8bc4b4be None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.860384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 279/555] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:29.861218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083007dd-4503-4bf2-9708-85c06ee87d88 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:29.861489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083007dd-4503-4bf2-9708-85c06ee87d88 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:29.861762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083007dd-4503-4bf2-9708-85c06ee87d88 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:29.863065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 280/556] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:29.867643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083007dd-4503-4bf2-9708-85c06ee87d88 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:29.886754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:29.937539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fc78d-29dd-42ca-a50a-5feb5f01c163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:29.939010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fc78d-29dd-42ca-a50a-5feb5f01c163 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:29.988025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083007dd-4503-4bf2-9708-85c06ee87d88 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.024738 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.025352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.026200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.026466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.027175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:30.027613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.029151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c5c74-c21b-4117-b96d-0843ef252dc2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.043971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.045959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 281/557] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:30.047025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.047331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.047605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.052375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 277/558] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:14:30.052375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.052375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.052375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.069107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:30.069270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:30.070299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:30.070422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:30.143337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 282/559] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:30.143337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.143580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 278/560] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:30.143580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.163094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.260912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96944c3-6b9b-47a8-bbd4-2a95e3ba3840 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.261147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675934c-f51d-491a-ae6f-f480ac8683a1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.289017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.297696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.298320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.298592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.298859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.302307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.302941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.303218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.303512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.309199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.310680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.311470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.312895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:30.325372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:30.326355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:30.332221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:30.332852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:30.333703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:30.334593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:30.334593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:30.334593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:30.334593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 279/561] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:30.334593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.334908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 283/562] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:30.334908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.405474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.441390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff7551-813f-4849-acdc-7a3348ddf0c5 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.449645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279f914-fcb1-4fc7-98d9-40dfe0626b88 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.479717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.480441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.480864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.481032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.488052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.488686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.488969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.489375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.492187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:30.497356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:30.498380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:30.499259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 284/563] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:30.499973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.504311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:30.509628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:30.510588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:30.511846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:30.515885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 280/564] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:30.516704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.563248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.611026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f592a33e-7a3b-47c5-9a11-049fffcbd855 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.626514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f660a4-f8a7-4d16-a5b0-f3164c23dceb None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.636318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.645126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.645731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.646008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.646277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.647003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:30.647332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.648756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.662568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.663110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.663427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.663703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.664342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:30.664647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.667338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.673512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 285/565] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:30.674559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.684467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 281/566] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:30.685324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.782630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9a44b-10b0-4bea-8c59-d6e02fc5df0e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.788346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.792454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30decd-8844-46f7-bf83-eefed3bfa34a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.819998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.820603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.820883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.821150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.821919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:30.822221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.824396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.828232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.828837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.829117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.829824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.830497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:30.830801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.832194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.832512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 286/567] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:30.833290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.844960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 282/568] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:30.844960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:30.882550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:30.936768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7057-ade5-469b-869c-fb08bc1bdc55 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.942910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f750ae-bfed-48cf-aa79-3a24a211d8fe None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:30.975325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.975948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.976222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.976488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.983664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:30.985085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:30.994810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:30.995455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.995736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.996012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ef745635-5b28-4381-ad44-72d43cdbf2e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:30.996163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:30.997156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:30.998070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 283/569] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:30.998658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34360-120c-4d3e-8210-3db29d8ee5ea None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:30.998943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34360-120c-4d3e-8210-3db29d8ee5ea None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:30.999204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34360-120c-4d3e-8210-3db29d8ee5ea None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.000433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 284/570] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:31.001112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba34360-120c-4d3e-8210-3db29d8ee5ea None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.011801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.013245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:31.027546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:31.028498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:31.029387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:31.029387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:31.029387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:31.029387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 287/571] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:31.029387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.033533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.110159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba34360-120c-4d3e-8210-3db29d8ee5ea None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.124597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.128359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee9e28-1a48-468b-a742-8da009a28124 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.142532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.143243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.143667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.143948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.154737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.156210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.164187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.164807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.165087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.165356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:31.169754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:31.170816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:31.171943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:31.171943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:31.171943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:31.171943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 285/572] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:31.171943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.176314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.177801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-5f39b543-ebcc-46b4-b8b4-dfc2cd719ee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:31.193022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:31.193942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:31.194904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:31.194904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:31.194904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:31.194904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 288/573] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:31.195609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.195609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.195869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.212896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:31.214677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:31.251453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.275233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491f109-ad5d-4538-9134-b1febf1984e0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.281762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 289/574] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:31.282592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.322465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.323062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.323361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.323630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.324304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:31.324601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.326012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.337559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 286/575] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:31.338969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.366364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.384599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9973286-f609-4343-95f7-4973cae373f9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.419698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.420299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.420578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.420856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.421492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:31.421789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.423163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.434893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 290/576] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:31.435552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.436178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10145e90-d6d2-43d7-9f87-30cd9842c0a3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.471290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.471828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.472109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.472382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.473015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:31.473315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.474678 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.476142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.486420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 287/577] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:31.488997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.537136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716cb9c-7bdc-495a-89f6-db24666ffcac None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.569701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.570295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.570569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.570838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.571497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:31.571795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.573663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.582017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.585200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 291/578] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:31.585965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.589432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc89ac8-22e0-4bbe-a6d4-db8ddd63e569 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.622758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.623406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.623681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.623947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.634177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:31.639715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:31.640700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 288/579] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:31.641553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.692102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39667cfd-8670-4bf6-a2d8-08d3f29ab304 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.695946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.727185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.727739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.728017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.728287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.729265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=aaf8d31c016449e6aa6408c7f69abe20) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:31.729577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.731001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.736291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 292/580] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/role_assignments?user.id=aaf8d31c016449e6aa6408c7f69abe20&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:31.737063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.741442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40649761-6356-49b4-8c9e-888bda0bb6df None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.782370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.783511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.783952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.784233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.793710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.795817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:31.809120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:31.810252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:31.811378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:31.817105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 289/581] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:31.817941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.820934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.872652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1edc99-8db9-44b8-a736-5d0fa785f676 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.910511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:31.911137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:31.911449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:31.911728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:31.920118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:31.921573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:31.933847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:31.938031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:31.939215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 293/582] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:31.940166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59ebec-1fec-493f-85e8-97307a7744c6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:31.940486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:31.977147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:31.999726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.000278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.000564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.000839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.010339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.011835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:32.023687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:32.024707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 290/583] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:32.027410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.048848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f11ea-1eca-4c98-876f-fabb68c3d684 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.073495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.084454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.085064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.091564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.091564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.091564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:32.091564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.091564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.101800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 294/584] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:32.102451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.136043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08112735-7767-47da-8f35-27ba9ca9a55f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.173761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.174373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.174656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.174931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.175657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:32.175961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.177402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.196562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 291/585] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:32.196562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.214970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2da34e-743a-4dbf-b691-f2009ef3c07d None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.233214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.251605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.252457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.252865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.253278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.254143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:32.254585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.256168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.275850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 295/586] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:32.276881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.300879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbdd583-aaa1-45ab-9ab0-4b53cfbc2c90 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.335821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.337875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.339324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.339719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.339991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.340709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:32.343315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.343315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.360126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 292/587] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:32.360878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.405496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6817f8-3e7d-49bb-ab34-6b64efa1ffce None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.441819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.442449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.442726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.442998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.454617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.456094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.456590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.465748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a2290-0bf9-47f5-aeda-30af5ec23bfc None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:32.472008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:32.472947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:32.473822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:32.473822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:32.473822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:32.473822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 296/588] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:32.474078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.505454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.506031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.506306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.506642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/c1031699d7da4a76965035faa9305dbd/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.521902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=c1031699d7da4a76965035faa9305dbd, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.523812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.573366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.578907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ac21b-f2ec-4895-9ca3-e00c76ab0f1e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.606550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 293/589] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:32 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/c1031699d7da4a76965035faa9305dbd/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:32.606550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.614096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.614919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.615390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.615791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.628420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.630178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:32.651919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:32.653222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:32.654310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:32.654310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:32.654310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:32.659468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 297/590] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:32.660655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.762645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfaba8-e05b-48b9-bfb9-0f519ea062dd None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.792154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235ebc-cfc5-4897-86de-f48c76fbdd59 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.814556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.815066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.815400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.815674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.828700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.830779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.832974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.833533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:32.834143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:32.834419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:32.834689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:32.835404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:32.835670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:32.837059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:32.850888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 298/591] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:32.851810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:32.852828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:32.855533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:32.856602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:32.856602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:32.856602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:32.856602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 294/592] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:32.861733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:32.871482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:32.971757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2157d-9537-49b1-8852-accb22e26016 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:32.975340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec796fe-67c9-4b09-ae21-ee0c276af9d9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.012494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.013481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.013820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.014477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.019270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.020063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.020492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.020912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.021693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:33.022137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.023667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.024967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.027329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.040321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 299/593] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:33.040491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:33.042378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 00:14:33.043606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:33.045170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 295/594] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:33.045303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.046253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.109415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.149545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b04b5-f997-4cbe-ab7e-5b5e248c7d0b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.154609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d669c-5f31-41b1-bc25-da844a2b8164 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.157746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.186396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.187289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.187711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.188134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.189011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:33.189458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.191409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.192818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.194143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.194143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.194143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.202028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.203626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.209180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 300/595] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:33.210077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:33.215690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:33.216738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:33.217838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:33.223248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 296/596] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:33.223248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.319197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be82a3-6cdc-4ec0-a8fa-a84865968afc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.326568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849186b-c497-44e2-a82c-7ade54e7247e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.342274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.360347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.361014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.361297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.361571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.362334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:33.362637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.364108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.367609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.368532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.369442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.369930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.370493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.378007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.380657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.392263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 301/597] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:33.392999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:33.393846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:33.394828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:33.395307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.398317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 297/598] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:33.399335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.508629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5cf2-d979-4fdb-a0bf-471c73b51166 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.509971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01398b0f-3836-4621-b737-b03ad79de8ae None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.553604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.554160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.554282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.554490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.554759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.555732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.556229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.557128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.558504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:33.559034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.560700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.567175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.568642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.584424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 302/599] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:33.585034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:33.585933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:33.586852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:33.586852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:33.586852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:33.589314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.591948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 298/600] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:33.594238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.634579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.656108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.714202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ecc87-42ea-4291-bdbe-249d8a178890 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.719834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff4d08-52f1-4f53-8005-2a546bd3d86f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.752928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.753538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.753817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.754093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.754804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:33.755091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.756570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.757608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6637151-8269-4537-8b03-08e29559753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.758487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637151-8269-4537-8b03-08e29559753c None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.758973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637151-8269-4537-8b03-08e29559753c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.759505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637151-8269-4537-8b03-08e29559753c None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/de6c52a7640e47c7be2253e32aa5a173/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.775192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 303/601] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:33.776363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.778333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6637151-8269-4537-8b03-08e29559753c None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=de6c52a7640e47c7be2253e32aa5a173, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.780086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6637151-8269-4537-8b03-08e29559753c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.863033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 299/602] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:33 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/de6c52a7640e47c7be2253e32aa5a173/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:33.868245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6637151-8269-4537-8b03-08e29559753c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.890920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad03c3e-d610-43fe-98ca-66f6a7a3829f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:33.900130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:33.933340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:33.934004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.934355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.934706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.935532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:33.935943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:33.937796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76116a45-c9cb-4188-b4b9-0f6dea38fc79 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:33.952128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 304/603] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:33.954136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d97559-f665-4552-b813-4b2950eb5109 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:33.961456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d97559-f665-4552-b813-4b2950eb5109 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:33.961456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d97559-f665-4552-b813-4b2950eb5109 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:33.961456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 305/604] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:33.962033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d97559-f665-4552-b813-4b2950eb5109 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:33.984463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:34.019950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6637151-8269-4537-8b03-08e29559753c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.059854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.060756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.061198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.061622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/2e66698136f4429b8df67e5050a8da2f/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.079830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=2e66698136f4429b8df67e5050a8da2f, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:34.081513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:34.096341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d97559-f665-4552-b813-4b2950eb5109 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.132928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.133504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.133790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.134056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.142389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:34.143872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-e220a9ea-4dcb-4e17-8503-21c2f0baad32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:34.155404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:34.156567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:34.157635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 306/605] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:34.158134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.158292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.163375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.179381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 300/606] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:33 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/2e66698136f4429b8df67e5050a8da2f/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:34.180538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:34.194817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:34.197168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:34.250263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:34.335698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 307/607] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:34.358454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:34.385427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e9a7d-2819-4ea0-aba5-b605122efba4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.421303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:34.433708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.435458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.435458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.435458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.436482 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:34.437371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:34.439277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:34.461719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 301/608] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:34.463062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:34.486314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26423e-139b-4def-9231-c4d73f196ad7 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.522089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.522989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.523445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.523714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.535143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:34.542258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:34.543265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:34.545049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 308/609] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:34.550222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:34.574168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:34.637568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6067f86-d861-4842-b002-6682e9629ff9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.711267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678a521-a2fe-49ad-b667-023436818b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.713351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a521-a2fe-49ad-b667-023436818b54 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.713686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a521-a2fe-49ad-b667-023436818b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.714011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678a521-a2fe-49ad-b667-023436818b54 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/6c8a8736fc89485d87f14e06e85e67c1/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.719588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6f91-1938-43e7-9a3b-0ef9318bc84a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.743164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678a521-a2fe-49ad-b667-023436818b54 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=6c8a8736fc89485d87f14e06e85e67c1, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:34.745633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678a521-a2fe-49ad-b667-023436818b54 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:34.771762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.772421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.776489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.776489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.776489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:34.776489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:34.776489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:34.784897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 309/610] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:34.790275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:34.857784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 302/611] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:34 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/6c8a8736fc89485d87f14e06e85e67c1/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:34.883346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:34.908677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045d6b8-063e-4b4f-a695-e0280c065042 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:34.916612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0189544-be0a-4519-b284-6fa552655aeb None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.916974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0189544-be0a-4519-b284-6fa552655aeb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.917178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0189544-be0a-4519-b284-6fa552655aeb None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.918893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 303/612] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:34.924145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.924429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.924707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.949453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:34.950107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:34.950452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:34.950730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:34.964452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:34.964646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:34.965833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:34.965964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:34.984282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:34.985283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:34.986204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:34.986204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:34.986204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:34.986204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 310/613] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:34.993426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.012050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:35.070082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 304/614] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:35.100988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.156725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307f43e9-8973-401e-998e-4ad9cd50f67e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.191965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:35.192648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:35.193019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:35.193386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:35.194122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:35.194574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:35.196151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:35.210650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 311/615] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:35.211293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:35.216487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.231848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8c22aa-1e75-492f-b6d5-aa430e156bbb None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.282496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:35.284123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:35.284506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:35.285527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:35.307067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:35.313714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:35.314883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:35.316437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:35.319700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 305/616] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:35.326520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.398771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5b716-3cac-4303-8377-c7bf560b20cd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.438193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:35.473103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:35.473667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:35.473940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:35.474206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] PATH_INFO: `/v3/users/swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:35.495192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:35.497067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:35.518567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:35.519629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:35.520608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 312/617] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:35.528486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.543381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f18036-fde3-4727-a634-9e93678f77e5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.617757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:35.618410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:35.618685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:35.618954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:35.620173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:35.620289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:35.620532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:35.627662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:35.644622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 306/618] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:35.652869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.727329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9981ef-1299-4bab-b0e7-6f0d28870b5f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.797566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43272ac-5130-476a-8942-74d29559568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:35.804139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43272ac-5130-476a-8942-74d29559568c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:35.804661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43272ac-5130-476a-8942-74d29559568c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:35.805173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43272ac-5130-476a-8942-74d29559568c None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:35.806222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43272ac-5130-476a-8942-74d29559568c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:35.807334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43272ac-5130-476a-8942-74d29559568c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:35.809047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43272ac-5130-476a-8942-74d29559568c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:35.837720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:35.848568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 313/619] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 00:14:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:35.854924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43272ac-5130-476a-8942-74d29559568c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:35.859782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5360069-c7d9-434d-98a0-b37dac3f6b5e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.936298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:35.937437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:35.937933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:35.938384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:35.964270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:35.966068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:35.982381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43272ac-5130-476a-8942-74d29559568c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:35.989223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:35.990176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:35.991070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:35.991070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:35.991070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:35.991070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 307/620] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:35.997198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.008237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.033634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.034930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.035240 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.035529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.049672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.051153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:36.075499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:36.076500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:36.077390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:36.077390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:36.077390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:36.081272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 314/621] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:36.082454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.154475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c25f3-0b26-4c73-8079-a756deda2f38 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.201049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.201672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.201951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.202258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.203624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:36.203935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.205794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a66ec93-a27e-4347-b1a6-247b2415d011 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.218171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.227321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 308/622] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:36.229318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7acbe-7b6e-4862-9a3c-0499f9b84446 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.229602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7acbe-7b6e-4862-9a3c-0499f9b84446 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.229872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7acbe-7b6e-4862-9a3c-0499f9b84446 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.231262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 309/623] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:36.235626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7acbe-7b6e-4862-9a3c-0499f9b84446 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.240788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae31e53-cd5f-4329-8d59-d990be9172b1 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.282160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.283217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.283756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.284296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.285171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:36.285737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.287400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df6731-2231-40bb-8578-535f65688a45 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.305814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 315/624] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:36.308127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.308622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.309209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.329182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:36.330760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:36.345697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7acbe-7b6e-4862-9a3c-0499f9b84446 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.348245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.380995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.381638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.381903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.382173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.389595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.391082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:36.403333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:36.404332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:36.406127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:36.409222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 310/625] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:36.409954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.420497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 316/626] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:36.421691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.473872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.512426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94e055-ce5d-4822-bb6f-0b2546366f72 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.547834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.548589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.548892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.549288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.554384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe517215-ebe5-470c-b55a-fa91efdfe28b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.558782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.560267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:36.574841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:36.575843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 311/627] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:36.581944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.602783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.604579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.605090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.605651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.606587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:36.607107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.609000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.629591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 317/628] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:36.631008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.643546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.689351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d7d8a-bf7a-4957-b370-321b7384f984 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.710826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.724984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.725614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.725900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.726169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.737082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:36.742202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:36.743132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:36.744031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 312/629] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:36.744927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.753736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92b5a3-1499-4e3c-9513-5bd81389fa1e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.794112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.794639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.795115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.795251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.795899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:36.796228 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.797622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a23ec-2c53-4f32-aa34-fbea8c5f5ab0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.813171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 318/630] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:36.814434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ed89e-85f9-4860-971c-53d64673d0bb None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.814767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ed89e-85f9-4860-971c-53d64673d0bb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.815039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ed89e-85f9-4860-971c-53d64673d0bb None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.817995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 319/631] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:36.817995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ed89e-85f9-4860-971c-53d64673d0bb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:36.860814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaef427-329a-4dea-a5c7-cc66c956d9df None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.894789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.896725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c596-e3bf-44fe-b599-11af666fa9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.897335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c596-e3bf-44fe-b599-11af666fa9a9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.897612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c596-e3bf-44fe-b599-11af666fa9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.897884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c596-e3bf-44fe-b599-11af666fa9a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.932518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ed89e-85f9-4860-971c-53d64673d0bb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:36.948946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:36.966180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:36.966790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:36.967064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:36.967365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:36.968028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:36.968327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.969684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.971423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c596-e3bf-44fe-b599-11af666fa9a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:36.973192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c596-e3bf-44fe-b599-11af666fa9a9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:36.977854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 320/632] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:36.979295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.079387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 313/633] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:37.081022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.081495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.081968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.094420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235243f-d3ce-412e-a90e-d8b651ee9163 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.100779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:37.101988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:37.123977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.129596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.130242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.130515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.130783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/aaf8d31c016449e6aa6408c7f69abe20/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.145850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=aaf8d31c016449e6aa6408c7f69abe20, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.147393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.169007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 314/634] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:37.170113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.180573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.252059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 321/635] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:36 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/aaf8d31c016449e6aa6408c7f69abe20/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:37.252924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.267276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e7681e-d8aa-4642-a73b-0e07777d2f87 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.304121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.304827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.305102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.305375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.317280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.318754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-2381641f-3c45-4cdc-a12c-07c8398fc246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:37.336566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:37.337563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:37.338517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:37.338517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:37.338517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:37.338517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 315/636] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:37.339576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adea68-da2a-471b-997f-93bc80d0b9c3 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.339858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adea68-da2a-471b-997f-93bc80d0b9c3 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.340131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adea68-da2a-471b-997f-93bc80d0b9c3 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.345449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 316/637] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:37.346343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adea68-da2a-471b-997f-93bc80d0b9c3 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.390220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6a36a-072e-4d06-9a2d-7958c879af43 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.418194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.433915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.434551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.434827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.435096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.446368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.449365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.462382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adea68-da2a-471b-997f-93bc80d0b9c3 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:37.464852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:37.465816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:37.466728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:37.466728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:37.466728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:37.466728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 322/638] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:37.466953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.502435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d3e9-42f0-4354-ad86-78ed73789ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.503115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d3e9-42f0-4354-ad86-78ed73789ecf None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.503432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d3e9-42f0-4354-ad86-78ed73789ecf None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.503704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d3e9-42f0-4354-ad86-78ed73789ecf None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.506959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.514310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d3e9-42f0-4354-ad86-78ed73789ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b060d3e9-42f0-4354-ad86-78ed73789ecf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:37.519899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:37.524876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 317/639] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:37.526039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.526950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.526950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.543057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:37.544287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:37.586065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c0aff-3f33-4114-bb5d-75eb61f04db3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.614927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 318/640] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:37.615828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.627400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.628010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.628294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.628566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.629212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:37.629512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.630871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.644243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 323/641] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:37.645087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.690779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.744832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19795c3-8fcb-4878-969d-088af41e11cd None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.752577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.763925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba87ada-a901-4360-80c5-cc0eaa9fc213 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.787348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.788263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.788687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.789101 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.789881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:37.790478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.792755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.805482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.806088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.806369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.806675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.807693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:37.808002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.809363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.809970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 319/642] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:37.810985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.823141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 324/643] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:37.823141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.922040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b060a-c748-4604-b1e3-bd911cadd685 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.925798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347681d-fe0f-4ff7-b316-4cc9fa315e0b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:37.936011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:37.957685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.958316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.958599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.958871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.962544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:37.963874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:37.964153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:37.964427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:37.969726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:37.973487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:37.974827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:37.975864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:37.977127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:37.977930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:37.978033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 320/644] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:37.978033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:37.989423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:37.990478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 325/645] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:37.996943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.012920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.084600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb39df-c7fa-4d74-8b16-19172c16cce7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.107716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4752194-330d-4b2d-bed7-a655e5f3dc52 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.118906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.119668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.119946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.120251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.127911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.129395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:38.142240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:38.148317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 321/646] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:38.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.184303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.257530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4866e-df70-4839-9548-d5ece9873735 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.261559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.261742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.261897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.262153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.272604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.274418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:38.291873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:38.292927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:38.293867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:38.293867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:38.293867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:38.293867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 326/647] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:38.293867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.301760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.304118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.304118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.304118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.304118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:38.304118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.306075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.315944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 322/648] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:38.316825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.368201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.405102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbda8f8-7be2-4f43-a23c-03b7ea71b930 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.424314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.431716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2f9fd-eacf-46d0-a314-19deaad8d858 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.440648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.441580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.441988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.442265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.442987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:38.443314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.444736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.469465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 327/649] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:38.469465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.471207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.471860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.472135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.472405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.473112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:38.473411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.474834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.487527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 323/650] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:38.488315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.575783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a52a97-2ec7-44b7-b327-4f1dba608a44 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.585159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b947ccc-31fb-41ac-9bab-e384884f5fb6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.601301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.612209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.613626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.613626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.613626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.614003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:38.614301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.615677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.620730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.621512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.622056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.622928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.627071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 328/651] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:38.627893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.636212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.637665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:38.657961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:38.659111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:38.660234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:38.660234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:38.660234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:38.663620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 324/652] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:38.664468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.676918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.742476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151716-1433-4c7f-b78f-914e86a2ca6b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.773864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5ce5-2b09-4076-92fd-5ea1a3ad2773 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.778575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.779183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.779490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.779762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.812861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.813458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.813981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.814254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.815850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=c1031699d7da4a76965035faa9305dbd) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:38.816418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.817861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.825951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 325/653] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/role_assignments?user.id=c1031699d7da4a76965035faa9305dbd&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:38.826767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:38.839460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.870128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:38.872037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:38.912340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:38.947733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de726b4-4b51-432c-98b9-198fe49abbcc None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:38.987264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:38.987896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:38.988334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:38.988615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:38.993065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 329/654] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:38.993772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.002312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.003807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:39.019108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:39.020109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 326/655] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:39.021627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.110357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.126127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7902e43-15c6-4a53-bda3-c99b5e65629b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.158729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9629ea-cd62-447f-b69f-8126fffbd46a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.164308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.165223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.165745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.166967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.167878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:39.168401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.169979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.182330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 330/656] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:39.183477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.193926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.203990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.204950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.205233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.205541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.220973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.222801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:39.242857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:39.243937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:39.244916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:39.244916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:39.244916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:39.246990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 327/657] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:39.252334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.306190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bacd9-1c31-4da1-802e-e692a9394554 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.343848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.344451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.344730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.345005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.345735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:39.346039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.347484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.362177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9505a16-dea5-49ed-9c9e-44f7875e74c0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.366034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 331/658] 10.222.0.15 () {56 vars in 1137 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:39.366857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.373511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.398675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.399380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.399648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.399960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.409304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.410481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e5fae7f4-2755-4b0d-95b2-93c6258dac58 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:39.422010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:39.422980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:39.423963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 328/659] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:39.424601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9b793-0de3-42ae-a738-e744e80eef4c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.425346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9b793-0de3-42ae-a738-e744e80eef4c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.425346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9b793-0de3-42ae-a738-e744e80eef4c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.426459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 329/660] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:39.431085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9b793-0de3-42ae-a738-e744e80eef4c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.480084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f445a-ad90-4c80-8285-d07334c4e034 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.481326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.514826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.515377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.515656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.515926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.516565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:39.516866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.518234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.531213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 332/661] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:39.532074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.544623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9b793-0de3-42ae-a738-e744e80eef4c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.583444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.584057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.584334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.584604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.605242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.645580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7254d-2011-4aa1-9dae-dbb93c956930 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.663728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.666814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.683941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.684577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.684853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.685124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.685849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:39.686147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.687629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab11283-8567-40e0-9125-5346252e4448 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.710654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 333/662] 10.222.0.15 () {56 vars in 1139 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:39.711923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.712207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.712481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.715559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.731686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:39.733160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:39.779939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 330/663] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:39.780880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.807399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 334/664] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:39.807399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.867376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.893430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408cee8-cb78-48be-87af-59bb7529277f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.922618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c50f8-d26f-4726-973d-af72f9d77d79 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:39.929583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.930205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.930525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.930802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:39.937759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:39.939185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:39.950964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:39.952038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 331/665] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:39.952969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:39.959110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:39.965201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:39.965811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:39.966125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:39.966438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.034860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.036235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.063065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d966715-a2f5-4ece-aeb5-5372bfa09ff3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.089476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.101325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.101952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.102222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.102487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.114315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.115802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:40.131034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:40.132038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:40.132948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:40.132948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:40.132948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:40.132948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 332/666] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:40.132948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.152084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 335/667] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:40.152941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.196421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.245459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4ebca-03e1-4544-9c61-8cdf189692ec None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.255148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3198a3-8de6-49de-a057-e1ab599cba51 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.281521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.282134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.282415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.282685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.288597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.289134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.289411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.289676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.290352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:40.290645 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.292108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.293841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:40.299326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:40.300265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:40.301139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:40.307273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 336/668] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:40.307273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.308590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 333/669] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:40.309433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.322818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.413599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6705be4-8ba5-45b7-aba8-692e92df4811 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.415184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8833f25-d537-4c14-a976-a5a2cfa40e92 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.430613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.451682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.452037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.452291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.452569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.457389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.457492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.457579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.457579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.457737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:40.457737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.457737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.464038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.465629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.476216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 337/670] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:40.476216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:40.481485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:40.482437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:40.483936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:40.483936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:40.483936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:40.483936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 334/671] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:40.483936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.551075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.584560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179fece-dfc6-4b22-a23d-356aa8249e64 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.590859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99996a13-5bea-4c42-ac1f-72c9a7711216 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.625423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.625947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.626070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.626294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.626541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.626654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.626819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.627080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.627269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:40.627525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.627862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=de6c52a7640e47c7be2253e32aa5a173) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:40.628168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.628884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.629653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.647527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 335/672] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/role_assignments?user.id=de6c52a7640e47c7be2253e32aa5a173&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:40.647527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.650458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 338/673] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:40.653433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.692081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.776182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430fcca-a2d7-42aa-946f-225c4050a1ea None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.805483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eb3740-6f46-48d5-ba0a-c8a443327e44 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:40.825715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.826369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.826626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.826918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.835289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.836723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.844148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:40.849747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:40.850618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:40.851546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 336/674] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:40.852049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.857648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:40.861083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:40.861083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:40.861083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:40.861083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:40.861083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:40.861419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:40.877005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 339/675] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:40.877887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:40.979560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:40.997322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309f180-779f-4028-a8b9-76a79ab21951 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.007121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689d70c-cbce-455a-a0f0-0312c83b8639 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.037459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.038348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.038778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.039052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.042336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.042937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.043213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.043519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.044266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:41.044568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.046040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.051210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.052677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.059668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 340/676] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:41.060495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-487ca88b-4227-40d3-ab0f-e9e511a2b5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:41.073002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:41.074147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:41.075207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:41.075207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:41.075207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:41.079273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 337/677] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:41.080563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfe0f-c081-490f-a916-ec91549c837f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.080842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfe0f-c081-490f-a916-ec91549c837f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.081114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebfe0f-c081-490f-a916-ec91549c837f None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.082393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 338/678] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:41.086947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebfe0f-c081-490f-a916-ec91549c837f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.121354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:41.176234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e08c50-0507-455e-a614-bb89c771d32c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.194720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebfe0f-c081-490f-a916-ec91549c837f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.214021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.214672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.214945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.215211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.222163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.223631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.231187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.231955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.232246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.232515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.233240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=2e66698136f4429b8df67e5050a8da2f) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:41.233544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.235276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-df42cfa8-fd53-4e58-9c10-08853f109d14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:41.237514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:41.238531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:41.239632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:41.255488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 339/680] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/role_assignments?user.id=2e66698136f4429b8df67e5050a8da2f&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:41.255718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 341/680] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:41.255837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.255927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.255927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.255927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.267345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:41.276555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:41.281338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:41.361518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 342/681] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:41.362837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.396352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65817f-ed5c-4d6c-a57b-5130dcdffe81 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.431781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:41.444109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.444874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.445945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.445945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.446643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:41.447998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.449086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.466563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 340/682] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:41.467789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.506554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04777d8d-252a-42f9-98c7-fe0e4548c18b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.542301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:41.552979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.554691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.554691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.554691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.555620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:41.556027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.558385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.579258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 343/683] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:41.579258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.585082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf858d-6693-4bbd-9ba4-8aa4916b24b2 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.621658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.622573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.623040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.623498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.634488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:41.639701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:41.640634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:41.641473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:41.644146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 341/684] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:41.645130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.721451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d045a6-1475-430e-9599-a1a1808ac98c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.723414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:41.756987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0a9c9-8753-4f3e-9702-3baad40ceae5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.767584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.768651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.769198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.770146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] PATH_INFO: `/v3/users/swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.780238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.782557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.787034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:41.796849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:41.798042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:41.798415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:41.800897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:41.801871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:41.802423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:41.802423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:41.802560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 344/685] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:41.802629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=6c8a8736fc89485d87f14e06e85e67c1) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:41.802697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.803638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:41.805553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:41.819543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 342/686] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/role_assignments?user.id=6c8a8736fc89485d87f14e06e85e67c1&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:41.820803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:41.959543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebe8f6-a254-4615-9abc-154f102657fd None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:41.999653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.000390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.000835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.001178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.001975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:42.002348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.004008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.004671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c5a15-e48a-4d67-9fdb-54d8d6c70efa None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.023641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 343/687] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:42.029414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.070490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.071813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.072852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.074458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.074458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:42.074458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.076255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.091259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 345/688] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:42.094587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.124710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:42.125611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:42.180179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a02fe1-30bf-49b8-ab0b-04e96e5f1c34 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.213308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d85e9-673c-4ee1-aff9-ef23bfe073ee None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.219112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.219112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.219533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.219904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.252339 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.253438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.253924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.254443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.266052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.267838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:42.284424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:42.285422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:42.286316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:42.286316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:42.286316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:42.288535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 346/689] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:42.291926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.294313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.294978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.347910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:42.402228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 344/690] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:42.422927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.423072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:42.469679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c62048-38a4-4e48-b558-3a4c7f9a63cf None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.515154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ab437-2056-4ae8-8a29-be60620e9fc5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.545806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.546485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.546863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.547141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.547913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:42.548217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.549577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.552874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.553632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.554008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.554379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.565250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.567087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.573758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 347/691] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:42.578876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:42.582661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:42.583851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:42.585076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:42.585076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:42.585076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:42.586378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 345/692] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:42.587341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:42.594401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.740536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b48734-68e3-45c5-a653-464c99850a58 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.783131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.783773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.784051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.784328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.785022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:42.785327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.786713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.805841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2823a6-a517-40c1-a139-bda742920d30 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:42.817085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 346/693] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:42.818023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:42.868535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:42.869785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:42.870058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:42.870319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:42.875829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:42.885407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:42.887819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:42.914241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:42.915314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:42.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:42.922972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 348/694] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:42.925111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.086762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.133691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de248c-6a6d-4dcb-a88e-2b16357c03e4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.146540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371416e0-d5cc-4d28-8fac-1dccd6159c27 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.181002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.182035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.182706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.183164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.183448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.183834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.184115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.184391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.185095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:43.185495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.187205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.192092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.193687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f8584d9b-47e4-4a29-89e2-011b869c32c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:43.205160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:43.206115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:43.207020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:43.209359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 347/696] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:43.210892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82881189-66a6-44ea-a9c1-1bc7893e67da None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.211423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82881189-66a6-44ea-a9c1-1bc7893e67da None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.211934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82881189-66a6-44ea-a9c1-1bc7893e67da None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.212565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 349/696] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:14:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:43.214330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.215773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 348/697] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:43.220635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82881189-66a6-44ea-a9c1-1bc7893e67da None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.282688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.333316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8d81d-bbfc-4de4-9bf7-c6a6b4920099 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.352243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82881189-66a6-44ea-a9c1-1bc7893e67da None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.371001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9638d-709e-440c-842e-64b2d1f78de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.371978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9638d-709e-440c-842e-64b2d1f78de1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.372464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9638d-709e-440c-842e-64b2d1f78de1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.372966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9638d-709e-440c-842e-64b2d1f78de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.376979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.395253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.395860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.396203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.396461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.397421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:43.397726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.399486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc7087-dec6-4293-bd2a-3064788bf0c5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.416028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 349/698] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:43.417367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.417655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.418013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.439982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:43.441656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:43.467731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9638d-709e-440c-842e-64b2d1f78de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.470007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9638d-709e-440c-842e-64b2d1f78de1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.527439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.528352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 350/699] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:43.529228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.596300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 350/700] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 00:14:43.597611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc831cf6-8ea1-48fb-a51d-4601812d6583 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.597898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc831cf6-8ea1-48fb-a51d-4601812d6583 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.598174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc831cf6-8ea1-48fb-a51d-4601812d6583 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.599437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 351/701] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:43.600127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc831cf6-8ea1-48fb-a51d-4601812d6583 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.641306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f67cf-871c-4248-a3eb-6a75d62e1ae1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.654971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.682080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.682650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.683140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.683419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.684317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=aaf8d31c016449e6aa6408c7f69abe20) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:43.685082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.686624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.703254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 351/702] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/role_assignments?user.id=aaf8d31c016449e6aa6408c7f69abe20&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:43.703254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.729929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc831cf6-8ea1-48fb-a51d-4601812d6583 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.775887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.777032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.777614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.778251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.790340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.793132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.812314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a341a2-42a7-4e21-b840-58c12d932c25 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8faa76a2-2225-498a-8037-dd59a5c53501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:43.813414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:43.814315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:43.815324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:43.815324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:43.815324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:43.823991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 352/703] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:43.823991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.823991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.823991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.841997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.849526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:43.851614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:43.861054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:43.862550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:43.862550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:43.862550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:43.877505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:43.878967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:43.896927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:43.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:43.899018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:43.899018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:43.899018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:43.899018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 352/704] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:43.903345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:43.934279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:43.947902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 353/705] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:43.955418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.037344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24034ea5-da34-4c34-89a8-4272517e7631 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.077505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9ae245-4254-46f0-b832-296c08836551 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.097013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.098186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.098697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.099149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.100525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:44.101151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.103063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.115122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.115897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.116276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.116641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.117392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:44.117781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.120593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.124393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:44.130857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 353/706] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:44.132140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.142976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 354/707] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:44.147353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.263243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:44.283690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5d967-8062-489d-a83f-a56a4b5984e7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.289694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947d3946-bdcc-4ea5-aa03-fe1270479dde None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.333944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.334995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.335305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.336690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.338639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.340046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.345075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.346220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.346736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.347211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.348089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.349788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:44.351430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:44.352469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:44.353389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:44.357625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 354/708] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:44.358489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.459717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc0e69-341f-4148-bc3c-3a1f6cd5f312 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.461021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 355/709] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:14:44 2022] POST /identity/v3/users => generated 254 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:44.472484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.505618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.506370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.506712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.507051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.518097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.519982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-6ad259d3-3704-4391-b30d-6b0c7b1604ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:44.536002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:44.536981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:44.537904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:44.539370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 355/710] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:44.540978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf32a-e010-4698-a295-709cd5323f7d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.541601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf32a-e010-4698-a295-709cd5323f7d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.542026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bf32a-e010-4698-a295-709cd5323f7d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.545843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 356/711] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:44.546664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf32a-e010-4698-a295-709cd5323f7d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.557930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:44.568653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:44.630295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959dbd4-b670-44fd-80ba-09d3937f3fa7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.683449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.684064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.684336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.684605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.685241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:44.685539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.686901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae391b-b645-4344-98de-018a0b2b83e0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.690991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bf32a-e010-4698-a295-709cd5323f7d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.693355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 356/712] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:44.694844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.695366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.695851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.714976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:44.716462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:44.731144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.732552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.732552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.732552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.733051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:44.733358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.734796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.750899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 357/713] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:44.752117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.801048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 357/714] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:44.801906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.839793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:44.881023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96c2ea-b220-4d8d-8882-c74409bb3df6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:44.885092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:44.919344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:44.919953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:44.920226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:44.920494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:44.921120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:44.921417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:44.923125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:44.929677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 358/715] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:44.934939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:44.975013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e9bb5-c94a-4516-8ccd-d4f95de2cf8c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:45.046232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:45.047636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:45.048760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:45.050290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:45.051282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=c1031699d7da4a76965035faa9305dbd) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:45.052626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:45.054327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:45.066544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 358/716] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/role_assignments?user.id=c1031699d7da4a76965035faa9305dbd&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:45.126543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede8811-25ff-4bb6-878c-87c16e269a7e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:45.175643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:45.176799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:45.177281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:45.177799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] PATH_INFO: `/v3/services/network` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:45.178599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:45.181566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:45.186056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:45.196258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:45.199947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:45.200854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:45.203118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 359/717] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:44 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:45.209141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:45.332120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed4648-f636-47b0-a18c-d85dd4d131bb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:45.350762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:45.383933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:45.384566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:45.384848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:45.385112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:45.385739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] RBAC: Adding query filter params (name=network) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:45.386033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:45.387418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ed1ea-de1c-408e-990a-ce0755066a2d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:45.393740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 359/718] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:14:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:45.399349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:45.538235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35ad09-1134-43a3-bdd1-672446266b9a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:45.580129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:45.580812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:45.581090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:45.581360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:45.582042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] RBAC: Adding query filter params (type=network) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:45.582399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:45.583723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da1767-feb2-4f72-aba8-1a966f90ed5a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:45.596082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 360/719] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:14:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:46.109501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a836f-2c93-42d0-a86e-fb1d99dcc013 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:46.109882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a836f-2c93-42d0-a86e-fb1d99dcc013 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:46.110059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a836f-2c93-42d0-a86e-fb1d99dcc013 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:46.111709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 360/720] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:46.117483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:46.117775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:46.118105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:46.141877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:46.143056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:46.243917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:46.257993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 361/721] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:46.297843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a836f-2c93-42d0-a86e-fb1d99dcc013 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:46.433089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a836f-2c93-42d0-a86e-fb1d99dcc013 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:46.478959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:46.479618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:46.479892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:46.480156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:46.493304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:46.498668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:46.499653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:46.500507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:46.502493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 361/722] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:46.508113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:46.726870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae97d7-9a6c-44bd-aeb3-6ff97fe9eab7 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:46.765490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:46.766120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:46.766401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:46.766680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:46.767379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:46.767685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:46.769199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:46.778358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 362/723] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:46.788286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:46.838513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:46.929004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2432e-de5c-41ca-9718-bdf5f8bb6381 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:46.970396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:46.971038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:46.971350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:46.971643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:46.982133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:46.983555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-d595b9ab-72e8-47e2-aab0-4d843e47a28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:46.998750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:46.999730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:47.000706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:47.000706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:47.000706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:47.000706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 362/724] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:47.008582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.118269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:47.165337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347884-7620-4049-a7d7-61e7635a4635 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.165652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347884-7620-4049-a7d7-61e7635a4635 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.166435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347884-7620-4049-a7d7-61e7635a4635 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.169097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 363/725] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:47.173052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.173331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.173598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.183875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78fcea8-323b-4683-8150-f42f91756f94 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:47.192647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:47.193833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:47.237971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:47.238607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.238891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.239163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.239832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:47.240132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:47.241495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:47.267830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 363/726] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:47.274015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.291385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 364/727] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 00:14:47.339465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.367627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:47.449945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102dbe3-dc5c-4714-b299-2f043447b7b8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:47.459332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c6ceb-40e1-45cb-bd4e-835ed44f9ae4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:47.473303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:47.497769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:47.497769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.497769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.497769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.506728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:47.512148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:47.513104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:47.514012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 365/728] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:47.515219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:47.516086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.516526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.517018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.520567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.531208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:47.536836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:47.559750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:47.560770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:47.561819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:47.566809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 364/729] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:47.572847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.667587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:47.690296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e28341-9b89-40af-811c-1a15e326f784 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:47.703332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e732c3-33fe-47aa-bac7-0d77b0e15661 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:47.727989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:47.729022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.729136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.729361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.730021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:47.730317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:47.731704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee6c96-5d17-44c9-863c-a5fbe2760084 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:47.740414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 366/730] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:47.741717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbee2e-f7ce-4be6-b22e-99387593bb78 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.742002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbee2e-f7ce-4be6-b22e-99387593bb78 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.742267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbee2e-f7ce-4be6-b22e-99387593bb78 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.743434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:47.744883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.745415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.745915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.746680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:47.746991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:47.747368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 367/731] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:47.748666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2b361-46b6-4b6b-bc2a-5597798261e2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:47.748793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc179f5c-49e3-46ae-b6fe-7aa9cb8a2321 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.749035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc179f5c-49e3-46ae-b6fe-7aa9cb8a2321 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.749304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc179f5c-49e3-46ae-b6fe-7aa9cb8a2321 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.750872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 368/732] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:47.752270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc179f5c-49e3-46ae-b6fe-7aa9cb8a2321 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.770842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 365/733] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:47.772492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fccd6-d239-49e5-9ab2-46e0dc9ac884 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.772778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fccd6-d239-49e5-9ab2-46e0dc9ac884 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.773051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fccd6-d239-49e5-9ab2-46e0dc9ac884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.792845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1f2fccd6-d239-49e5-9ab2-46e0dc9ac884 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:47.794396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1f2fccd6-d239-49e5-9ab2-46e0dc9ac884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:47.821902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:47.867646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 366/734] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:47.869681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.869965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.870234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.886143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc179f5c-49e3-46ae-b6fe-7aa9cb8a2321 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:47.892865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:47.894108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:47.922896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:47.923847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:47.924197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:47.924464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:47.930918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:47.935971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:47.937490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:47.952855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:47.953823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:47.954735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:47.954735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:47.954735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:47.954735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 369/735] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:47.955410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:47.979320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 367/736] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:47.979320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.071095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca299b-b3e9-429f-81f6-a475e0e3f7c5 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.081003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.090133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add568dc-cae1-467b-b993-d5c4c7d00a9d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.111186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.111821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.112109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.112383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.128333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.129297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.129575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.129843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.141163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:48.145971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:48.146911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:48.147829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 368/737] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:48.148572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.182794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.194285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.203040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.259132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912e54d-1a81-48e6-b329-59710b42a937 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.298290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.298894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.299165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.299465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.300119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:48.300414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.301768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.307622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 370/738] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 00:14:48.308489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.318355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 369/739] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:48.318355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.331956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.401223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.410627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40607a7e-38d6-43b5-aec8-8621f908341f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.416798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823dcd6-de4c-4e5c-8aed-30326567e9bd None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.444698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.445298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.445597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.445862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.446492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.447883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.450412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.451036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.451348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.451627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.452281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:48.452593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.453960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698cb99-15b8-4e88-b930-7a621d74aae9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.462115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 370/740] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:48.463877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abfc7d-091f-44d0-8039-30e2cfbeb3f5 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.467074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abfc7d-091f-44d0-8039-30e2cfbeb3f5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.467074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abfc7d-091f-44d0-8039-30e2cfbeb3f5 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.467074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 371/741] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:48.467074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abfc7d-091f-44d0-8039-30e2cfbeb3f5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.538823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.551854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 371/742] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:14:48 2022] POST /identity/v3/services => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:48.553284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.577408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abfc7d-091f-44d0-8039-30e2cfbeb3f5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.621873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.622557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.622834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.623108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.635105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.636930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:48.657634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:48.658689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:48.659794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:48.659794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:48.659794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:48.659794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 372/743] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:48.659794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.690822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893b59b-ab67-484c-8c77-331e01a7fab1 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.732663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.734220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.734220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.734220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.744020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.746002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-47fbfad4-0d98-479a-b1ea-22993ecd514a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:48.758290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:48.759535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:48.760514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 372/744] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:48.761141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.761141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.761380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.774000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ace47f-caf9-4d1e-8aed-c4d31ffceec1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:48.778909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:48.780109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:48.782972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.797946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:48.809886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:48.810618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:48.810905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:48.811177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:48.822996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:48.824486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:48.838929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:48.839955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:48.840845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:48.840845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:48.840845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:48.844572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 373/745] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:48.845386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:48.855205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 373/746] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 00:14:48.856110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.047113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441b8d1-8337-4f4d-86ba-8a741ceee1a9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.051979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8b175-3348-4345-9c20-e4c3c1d2559f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.083491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.084103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.084390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.084685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.085326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:49.085630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.086996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.090705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.091345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.091626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.091895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.092622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:49.092924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.094373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.105374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 374/747] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:49.105374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.109867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.122897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 374/748] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 00:14:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:49.122897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.144440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.216017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5861-252b-4941-a24c-063e212bcfea None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.225715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97d15-7518-4e6e-b77e-2a0d31059136 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.251107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.252232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.252455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.252727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.253358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:49.253657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.255012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.261912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.262521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.262795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.263064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.263739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.265106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.271778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 375/749] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:49.272625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.363259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 375/750] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:14:49 2022] POST /identity/v3/services => generated 240 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:49.363259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.383282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.391158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078e673-0a26-4188-8a74-0e48dd0f263d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.436978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.437624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.437908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.438186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.444486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.448577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.450169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:49.463407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:49.464469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:49.465692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 376/751] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:49.466188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.503039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b215f-d2f9-42b9-a328-1be4bf79ee1a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.554045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.554686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.554930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.555990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.611511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c903f-4b61-4930-99cd-0fb5410ec650 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.643290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.645593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.648613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.650818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.651106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.651416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.660040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.661496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:49.676221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:49.677168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:49.678167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 377/752] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:49.678811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.719459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.737929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.790458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 376/753] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:14:49.791719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.800106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63a321-212c-4d41-8f96-47026e80a355 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.846000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.846929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.847851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.848329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.849200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:49.849685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.851817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.871507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 378/754] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:49.872702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.908579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135cdf-31e9-4de0-81b8-934a6a2ec51c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:49.955120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:49.955924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:49.956268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:49.956612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:49.957421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:49.957625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.957794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:49.959863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:49.978017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 377/755] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:49.978992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:49.998349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:49.998708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9e7f2e-c377-4537-9379-ff7475eaa9f0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.046271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.047262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.047945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.048432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.063359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.065301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:50.084792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:50.085784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:50.086817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:50.086817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:50.086817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:50.090743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 379/756] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:50.091758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.094925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8790ab-9b71-4b11-a858-9597d8052ddd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.145840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.146798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.147297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.147783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=aaf8d31c016449e6aa6408c7f69abe20) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:50.149875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.152185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.161530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 378/757] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:49 2022] GET /identity/v3/role_assignments?user.id=aaf8d31c016449e6aa6408c7f69abe20&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:50.162604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.245696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cbf64-68fa-49fe-b367-a269e8a97ad9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.268228 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:50.292948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ebe3a-677a-44ed-afba-8baf8bdaee92 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.299027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:50.336735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.336991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.337527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.338095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.338276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.338543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.338902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.339730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.340660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:50.341160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.342923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.356998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 379/758] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:50.363345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.366264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.368525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:50.393874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:50.394912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:50.395838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:50.395838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:50.395838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:50.399146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 380/759] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:50.400954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.484440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951b220-343b-47bb-ba64-acfc18dcf4f2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.581306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.581984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.581984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.582301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.589991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41a77d-32a9-492e-8201-0f0d0249a23a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.600502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.602166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.612550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:50.634816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:50.635811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 380/760] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:50.636727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.645375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.646002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.646282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.646554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.647205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:50.647584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.649236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6728bf0-6d16-4783-a4ce-e29a492944c5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.672736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 381/761] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:50.674801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a9f51-258d-4f92-9670-b3512f81a735 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.675356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a9f51-258d-4f92-9670-b3512f81a735 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.676056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a9f51-258d-4f92-9670-b3512f81a735 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.678500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 382/762] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:50.679511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a9f51-258d-4f92-9670-b3512f81a735 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.725478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:50.761593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5938436-e3ec-43f7-a237-2bc8b93cfac8 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.801773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.802385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.802663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.802947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.803664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:50.803968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.805515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e9872-f2a2-4e56-b2ef-582c78dbf0c6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.813517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a9f51-258d-4f92-9670-b3512f81a735 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:50.820621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 381/763] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:50.821943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.822431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.822803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.842652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:50.843984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:50.859159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:50.859865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:50.860147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:50.860417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:50.872643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:50.875006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:50.892153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:50.893088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:50.894027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:50.896917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 383/764] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:50.898439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:50.913872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:50.931455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 382/765] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:50.932759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.019422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:51.027554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ec040-a26f-4d63-a5cf-c910ec43f504 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:51.061984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:51.062599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.062881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.063160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.063915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=91ca9da53efa4421a21d5a824e25a483) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:51.064226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:51.065664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:51.068975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b74b6-0ff4-460b-a147-3d8c89b0a68c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:51.071773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 384/766] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/role_assignments?user.id=91ca9da53efa4421a21d5a824e25a483&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:51.073867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.112094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:51.112726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.113008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.113282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.114005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:51.114330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:51.115926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:51.130285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 383/767] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:51.139017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.250717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:51.275723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efaad6-86af-4020-802e-feaf0f8c16ce None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:51.321192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354c16d-7744-45a6-a636-857764473917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:51.321957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354c16d-7744-45a6-a636-857764473917 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.322971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354c16d-7744-45a6-a636-857764473917 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.323497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354c16d-7744-45a6-a636-857764473917 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.323868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354c16d-7744-45a6-a636-857764473917 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:51.325265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354c16d-7744-45a6-a636-857764473917 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:51.327851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3601c76-33c6-46a2-b5cf-a0736c210002 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-6354c16d-7744-45a6-a636-857764473917 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:51.343127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:51.344203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:51.345040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:51.346649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 385/768] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 8 00:14:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:51.353854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354c16d-7744-45a6-a636-857764473917 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.397434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b2e94-7e6a-498d-bcea-bb70a828b6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:51.398102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2e94-7e6a-498d-bcea-bb70a828b6b0 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.398386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2e94-7e6a-498d-bcea-bb70a828b6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.398664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2e94-7e6a-498d-bcea-bb70a828b6b0 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/c1031699d7da4a76965035faa9305dbd/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.405569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b2e94-7e6a-498d-bcea-bb70a828b6b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=c1031699d7da4a76965035faa9305dbd, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:51.407024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b2e94-7e6a-498d-bcea-bb70a828b6b0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:51.413316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:51.500270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354c16d-7744-45a6-a636-857764473917 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:51.541870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 384/769] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:51 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/c1031699d7da4a76965035faa9305dbd/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:51.548599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:51.548800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbb343-248e-4123-a483-6ef0f2621c1d None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.548978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbb343-248e-4123-a483-6ef0f2621c1d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.549217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.549320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbb343-248e-4123-a483-6ef0f2621c1d None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.549493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.549764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.550397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:51.550694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:51.550936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 385/770] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:51.552136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:51.560111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.560411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.560669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.561614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 386/771] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 00:14:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:51.569164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.585378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:51.586671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:51.737329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 386/772] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:51.751781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:51.764316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.768488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a3d74-3c51-4549-8919-5e01e748fd38 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:51.820997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:51.821655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:51.821982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:51.822473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:51.823279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:51.823602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:51.824964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:51.825523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:51.831844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 387/773] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 00:14:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:51.837601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:51.934074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37950718-5a81-4584-8db6-b9183c2bf06b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.000000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.000638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.000912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.001179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] PATH_INFO: `/v3/services/network` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.001763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.003301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:52.027468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:52.028707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:52.029705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 387/774] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:51 2022] GET /identity/v3/services/network => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:52.036521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.042994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1a228-d329-49e0-b1da-20e6595a171c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.106342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.107002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.107304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.107576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.108271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] RBAC: Adding query filter params (interface=public, service_id=418856fa37024d3fadb8d4818259ae67, region_id=RegionOne) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:52.108571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.109927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.120364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 388/775] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:14:51 2022] GET /identity/v3/endpoints?service_id=418856fa37024d3fadb8d4818259ae67&interface=public®ion_id=RegionOne => generated 194 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:52.125824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.143288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:52.243202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b79fe-a990-445e-8676-f92f572b5438 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.263604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:52.307678 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894ba471-460e-4e02-a4ed-effe1f5d9b91 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.312999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.313615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.313889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.314347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.314991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] RBAC: Adding query filter params (name=network) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:52.315360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.316984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.328395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 388/776] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:52.334002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.373134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25380-b80a-4e0e-83a9-4d2820521354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.374450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25380-b80a-4e0e-83a9-4d2820521354 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.374931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25380-b80a-4e0e-83a9-4d2820521354 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.375427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25380-b80a-4e0e-83a9-4d2820521354 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.376250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25380-b80a-4e0e-83a9-4d2820521354 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.383247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25380-b80a-4e0e-83a9-4d2820521354 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.390080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 389/777] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/services => generated 772 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:52.472172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99244827-633a-4aea-a5bc-49116537a057 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.473185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87360ae5-b1c8-438e-bebe-8b08b18ae260 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.473679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87360ae5-b1c8-438e-bebe-8b08b18ae260 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.474156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87360ae5-b1c8-438e-bebe-8b08b18ae260 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.476147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 390/778] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:52.482477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.482779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.483024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.511765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.512440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.512771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.513047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.513735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] RBAC: Adding query filter params (type=network) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:52.514034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.515487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.522929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 389/779] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:52.532876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:52.534031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.534321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:52.605923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:52.631079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 391/780] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:52.666780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.679288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:52.696917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501c29a-8e09-42c7-8253-2255f67cc458 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.734534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.735522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.735803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.736146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.736811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] RBAC: Adding query filter params (interface=public, service_id=ee6d1e108f214ab4b663f7f5917073b5, region_id=RegionOne) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:52.737214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.738575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.747057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 390/781] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/endpoints?service_id=ee6d1e108f214ab4b663f7f5917073b5&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:52.752453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.776227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581de91d-9b6c-4c9d-a031-2c47cb1b2792 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.826692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.827365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.827643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.827910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.840068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:52.845039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:52.845969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:52.846828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 392/782] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:52.851969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:52.853318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:52.858528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb931ea-509d-4713-8a93-b93d0d02f190 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:52.908922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:52.909456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:52.909732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:52.910001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:52.910602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:52.912125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:52.927749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:52.928411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 391/783] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:52.965306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad146868-f539-4856-8103-4510177a17d6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:53.004515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:53.005135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.005412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.005682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.006353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:53.006657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:53.008151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:53.018756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 393/784] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:53.024434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:53.159784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810dc19-1620-42e9-b029-b1d198647f21 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:53.195642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:53.196273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.196549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.196821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.208402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:53.209929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:53.225648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:53.226616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:53.227557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:53.227557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:53.227557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:53.227557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 392/785] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:53.233884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:53.362230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27929a2-3236-4850-9086-ab1593755509 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:53.400921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:53.406587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:53.407212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.417711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.417711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.417711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:53.417711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:53.417711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55700686-9b0f-4bd6-b23e-cc1146621574 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:53.427057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 394/786] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:53.436307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:53.482055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:53.579183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a32997-7277-40db-a6d8-06c35af9a1c7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:53.600570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecab93-40a9-46a3-b54f-ca83e1e56550 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.600874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecab93-40a9-46a3-b54f-ca83e1e56550 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.601128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecab93-40a9-46a3-b54f-ca83e1e56550 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.602707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 395/787] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:53.608307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.608590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.608866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.615846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:53.616491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.616750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.617024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.627467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:53.628212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:53.628924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:53.629606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:53.645632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:53.646643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:53.647616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:53.663246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 393/788] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:53.663246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:53.749454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:53.769568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b7d73-44fe-401b-901e-521a00c3ac53 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:53.782091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 396/789] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:14:53.806183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:53.807014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.807324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.807603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:53.808343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:53.808652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:53.810094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:53.828304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:53.831513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 394/790] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 00:14:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:53.834669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:53.943189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7d89b-fa44-463b-98eb-99a5104011c3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:53.979622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:53.988759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:53.993502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:53.994094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:53.994537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.012507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.014870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-daada9a9-3ab8-4fc5-a6ef-cd2594be598a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:54.044118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:54.045173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:54.046087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:54.046087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:54.046087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:54.049385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 395/791] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:54.055967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6929eb-ebc4-44ff-88f0-e261114cac03 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.057011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52be45-470b-40fa-806d-ba8bd12d822a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.057301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52be45-470b-40fa-806d-ba8bd12d822a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.057581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52be45-470b-40fa-806d-ba8bd12d822a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.059570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 396/792] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:54.060386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52be45-470b-40fa-806d-ba8bd12d822a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.083726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.099783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97dd9-2a98-4d41-a9e4-cb1507306353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.100566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97dd9-2a98-4d41-a9e4-cb1507306353 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.101014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97dd9-2a98-4d41-a9e4-cb1507306353 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.101377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97dd9-2a98-4d41-a9e4-cb1507306353 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.112431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d97dd9-2a98-4d41-a9e4-cb1507306353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a5d97dd9-2a98-4d41-a9e4-cb1507306353 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:54.117627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:54.118853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:54.119896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:54.120547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 397/793] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:54.125322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.125322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.125322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.141551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:54.142873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:54.174310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52be45-470b-40fa-806d-ba8bd12d822a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.219997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 398/794] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:54.220936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.227451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.228226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.228682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.229046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.229788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:54.230183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.231692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.257598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 397/795] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:54.262768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.323341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.334903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865bce4-fd6e-4005-a6ca-c8a87ae5a34d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.346152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.378695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.379335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.379618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.379891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.380553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:54.380851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.382216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.392114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 399/796] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:54.398870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.398997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d946ca-aaf8-468e-9aa4-836f902d853a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.440252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.440883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.441214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.441755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.454238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:54.459845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:54.461908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 398/797] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:54.464741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.514253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a02c66-8cb8-4bcd-82a1-9892aac1a068 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.551609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.552239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.552518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.552789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.559974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.561461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:54.573445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:54.575505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:54.577579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:54.583448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 400/798] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:54.583448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.595025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc5e01-1c85-4aaa-89b1-4e721c59c641 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.597100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.615653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.636771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.638020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.638461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.638949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.653166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.654911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:54.673384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:54.674418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:54.675433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:54.675433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:54.675433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:54.677806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 399/799] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:54.679059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.696175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aad0f1-75df-4211-bc64-56e62b7852e7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.738956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.739989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.740269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.740536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.741190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:54.741490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.743153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.755301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 401/800] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:54.756478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.798648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221563f-a5f3-421c-8c28-99b122b7347c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.836429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.837513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.838541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.839274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.840265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:54.840769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.842586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.861362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 400/801] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:54.862718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.875131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.881249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:54.882656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5b955-3d28-4ead-9591-645848d38f53 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:54.926954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:54.927585 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:54.927861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:54.928131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:54.929097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:54.929400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:54.931181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:54.948096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 402/802] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:54.949318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:54.979590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fcee0-fbf2-43d7-93ae-d5c03ffb780e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.020350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.021498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.021989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.022459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.034519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.036163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:55.051304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:55.052928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:55.054162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:55.054162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:55.054162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:55.055647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 401/803] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:55.055647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.069102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49539b8d-91db-4b2f-93ca-8ea026876934 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.103104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfd205-f645-4129-8551-19f1fff2861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.103799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfd205-f645-4129-8551-19f1fff2861e None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.104059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfd205-f645-4129-8551-19f1fff2861e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.104337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfd205-f645-4129-8551-19f1fff2861e None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/aaf8d31c016449e6aa6408c7f69abe20/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.106244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfd205-f645-4129-8551-19f1fff2861e None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=aaf8d31c016449e6aa6408c7f69abe20, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.107696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dfd205-f645-4129-8551-19f1fff2861e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.124516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.159031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896e52a-0d11-493b-a307-3ee62a1f1a11 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.187263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 403/804] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:54 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/aaf8d31c016449e6aa6408c7f69abe20/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:55.187263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037ec90-6703-49a5-a1d3-b74af2815404 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.187263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037ec90-6703-49a5-a1d3-b74af2815404 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.187263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037ec90-6703-49a5-a1d3-b74af2815404 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.187263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 404/805] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:55.187263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037ec90-6703-49a5-a1d3-b74af2815404 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.206965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.207262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.208468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.208799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.209124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.218701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.220433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-d3ce42e8-7899-48de-a73a-df1cbdce3f42 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:55.233296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:55.234310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:55.235328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 402/806] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:55.236440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709678-2995-4444-baab-20167520e34d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.236675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709678-2995-4444-baab-20167520e34d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.236951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc709678-2995-4444-baab-20167520e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.257740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-dc709678-2995-4444-baab-20167520e34d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:55.258969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-dc709678-2995-4444-baab-20167520e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:55.329378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037ec90-6703-49a5-a1d3-b74af2815404 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.349255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 403/807] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:55.350712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709678-2995-4444-baab-20167520e34d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.373729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.374342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.374677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.374993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.375701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:55.376004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.377479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec730a-ce9c-46c4-8beb-a071963429be None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.392421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 405/808] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:55.393740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496460df-d8db-4f34-8460-12224f54aa60 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.394024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496460df-d8db-4f34-8460-12224f54aa60 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.394354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496460df-d8db-4f34-8460-12224f54aa60 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.395857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 406/809] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:55.396559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496460df-d8db-4f34-8460-12224f54aa60 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.425563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.469853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc709678-2995-4444-baab-20167520e34d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.484045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.503357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.503965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.504241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.504511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.505234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:55.505530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.506942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54bcf1-b0cd-418b-b98d-0695188d122a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.512030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496460df-d8db-4f34-8460-12224f54aa60 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.524792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 404/810] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:55.526915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919715a-7383-4282-baef-6724b2213986 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.527197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919715a-7383-4282-baef-6724b2213986 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.527516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919715a-7383-4282-baef-6724b2213986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.546436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-5919715a-7383-4282-baef-6724b2213986 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:55.547646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5919715a-7383-4282-baef-6724b2213986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:55.558457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.559937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.560214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.560482 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.561050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.562400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:55.571510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:55.573418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:55.574863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 407/811] 10.222.0.15 () {56 vars in 1129 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:55.575998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.624347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 405/812] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:55.625193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919715a-7383-4282-baef-6724b2213986 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.675323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.694756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f8784-3aea-497c-aae3-40f6e52e7eae None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.731435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.732153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.732434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.732703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.736897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919715a-7383-4282-baef-6724b2213986 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.740355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.741780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.743606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:55.753612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:55.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 408/813] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:55.756140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.781044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.781809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.782085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.782352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.793287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.794740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:55.812300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:55.813432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:55.814611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:55.814611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:55.814611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:55.818141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 406/814] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:55.818968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.861042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737dc5b5-7df7-4970-9a45-1137c6cebcf8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.899307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.899844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.900315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.900968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.901604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:55.901907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.903666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.908782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 409/815] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:55.909395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:55.926154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.932416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3cbb33-aac7-4570-a07d-d7cc30c9a30a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:55.969221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:55.969958 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:55.970416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:55.970628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] PATH_INFO: `/v3/services/network` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:55.971487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:55.973416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:55.980993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:55.981981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:55.982975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:55.983843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 407/816] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:55.985138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.027992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633581e-7b0d-47bf-a71b-db89f7119b71 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.061935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.062530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.062808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.063079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.063825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:56.064194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.065612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.083694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 410/817] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:56.084488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.099955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e510f1-c0aa-48af-909d-93e1e32ad744 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.136170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.136768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.137043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.137311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.137943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:56.138236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.139626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.151203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 408/818] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:56.151956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.174989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.192036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bb3e9-d92e-4777-9943-f407885b3908 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.213360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.228861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.229548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.229891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.230230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.230917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:56.231700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.233522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.239090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 411/819] 10.222.0.15 () {56 vars in 1124 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:56.239964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.262986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f532361-682a-4ce7-9529-edc32ffe0318 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.299198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.299925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.300271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.300614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.301300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] RBAC: Adding query filter params (name=network) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:56.301684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.303135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.312838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 409/820] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:56.312838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.346693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4994d54-6004-4c00-adea-6c7df529237b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.381728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.382567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.382999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.383283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.409604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.426070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab76117-ed80-497c-b224-452c74b160d2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.436742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.453311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.454791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.467345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.468001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.468274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.468543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.476661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.478122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:56.490281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:56.491147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:56.492076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 410/821] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:56.492584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.563926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 412/822] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:14:56.565100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.598986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd398044-21e4-4e36-854a-f510b1ee1ad2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.631716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.632310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.632582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.632847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.633495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.635428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.653667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.668887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c6796-6253-4174-9b3c-c6ee6bca05ef None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.702123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.702719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.702991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.703280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.703897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] RBAC: Adding query filter params (type=network) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:56.704193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.705538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.710283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 413/823] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:56.710831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.759278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.775253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 411/824] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:14:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:56.775253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.836849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f1b9d-c807-438e-ab54-a4cbd1d37bf6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.890194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bd07b-7766-4a07-ae2e-9632e8e057bb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:56.894639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.899272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.899272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.899272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.899272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:56.899272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.899272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.915510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 414/825] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:56.917156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:56.935774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:56.942522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:56.943171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:56.944433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:56.944554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:56.957062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:56.958551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:56.974603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:56.976560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:56.977769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:56.977769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:56.977769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:56.977769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 412/826] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:56.977769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.032834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:57.062105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1812cc-7d2b-43e5-9b55-44d2d8d085ba None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.093880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb7d5-cc55-4dc8-aee1-15ab2618d971 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.101950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.102822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.103846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.104454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.105312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:57.106912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:57.131017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691539ec-2d06-4b3e-ab61-c7705a3782bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.131696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691539ec-2d06-4b3e-ab61-c7705a3782bb None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.132490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691539ec-2d06-4b3e-ab61-c7705a3782bb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.132490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691539ec-2d06-4b3e-ab61-c7705a3782bb None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/91ca9da53efa4421a21d5a824e25a483/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.148340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691539ec-2d06-4b3e-ab61-c7705a3782bb None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=91ca9da53efa4421a21d5a824e25a483, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:57.149868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691539ec-2d06-4b3e-ab61-c7705a3782bb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:57.187875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 415/827] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:14:56 2022] POST /identity/v3/endpoints => generated 337 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:14:57.188989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.252037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 413/828] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:14:56 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/91ca9da53efa4421a21d5a824e25a483/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:14:57.262970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed2cdd-b7a9-4bf2-ba4e-70a37501ff8e None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.263351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed2cdd-b7a9-4bf2-ba4e-70a37501ff8e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.263525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed2cdd-b7a9-4bf2-ba4e-70a37501ff8e None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.265033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 414/829] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:57.266146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:57.271272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.271831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.272316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.293493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:57.295125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:57.325208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76550f-0c0c-424e-8f84-d423bb913767 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.393773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 415/830] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:14:57.397035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.397775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.398152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.398523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.399307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:57.399703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:57.402197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:57.416565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 416/831] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:57.424544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.439124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.482548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:57.569004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48698050-38fa-42a4-982c-d4ba5f00d758 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.575854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9c29-2217-4996-a868-25fc9927d4b0 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.595202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:57.614477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.615869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.615869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.615869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.617541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.618254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.618623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.619562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.630053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:57.630313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:57.631980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:57.636292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:57.637243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:57.638082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 417/832] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:57.644099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:57.652261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:57.653193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:57.654114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:57.656661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 416/833] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:57.662380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.772571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:57.785793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da1cad-b82a-4941-a4cf-aa67d11106eb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.798960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c3715-b8bc-4e78-a593-1f8f077fad71 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.833087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.833698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.833977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.834249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.834910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:57.835212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:57.836602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:57.840650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:57.841270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:57.841543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:57.841809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:57.842505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:57.842802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:57.844243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:57.853527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 418/834] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:57.854879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.858299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 417/835] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:14:57.864134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:57.903422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:57.965744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb7001-62d9-420c-a007-03cd9047263a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:57.971677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bcd929-6207-44c3-b5b1-2b5a5d998c45 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:58.002418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:58.003424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.003636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.004005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:58.014945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:58.016057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.016438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.016812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:58.017028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:58.017752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=c1031699d7da4a76965035faa9305dbd) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:58.018184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:58.018949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:58.020020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:58.027974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 418/836] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/role_assignments?user.id=c1031699d7da4a76965035faa9305dbd&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:58.035523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:58.036534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:58.037450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:58.037450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:58.037450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:58.039710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 419/837] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:58.048078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:58.064705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:58.167757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b12fa6-f20c-4ada-9c49-2e0bdd9f9f5b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:58.206103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:58.207069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.207576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.208022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:58.208828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:58.209307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:58.210843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:58.223518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 419/838] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:58.230068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:58.332996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:58.340299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de1e45-8e96-4c8a-b945-33e3a17f52df None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:58.375924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:58.376640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.376985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.377323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:58.390224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:58.391912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:14:58.404062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:58.405255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 00:14:58.406453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:58.408681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 420/839] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:14:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:58.414602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:58.492686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:58.524529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55e80-38ce-41ba-bd55-b64ab2c8debb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:58.560450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:58.561060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.561337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.561605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:58.562339 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:58.562641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:58.564088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:58.583083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 420/840] 10.222.0.15 () {56 vars in 1135 bytes} [Mon Aug 8 00:14:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:58.589399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:58.706106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6cc92-cd35-4b4e-99ae-eda7959b6bc4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:58.741707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777c256-c447-4f6c-9399-4e99e054709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:58.742686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c256-c447-4f6c-9399-4e99e054709a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.743217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c256-c447-4f6c-9399-4e99e054709a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.744271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777c256-c447-4f6c-9399-4e99e054709a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:58.770479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:58.815455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777c256-c447-4f6c-9399-4e99e054709a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:58.817518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777c256-c447-4f6c-9399-4e99e054709a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:58.922213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 421/841] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:14:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:14:58.927867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:58.997158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03449cf-b7b1-4ba9-baa9-5eeb24d526fa None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:58.997510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03449cf-b7b1-4ba9-baa9-5eeb24d526fa None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:58.997714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03449cf-b7b1-4ba9-baa9-5eeb24d526fa None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.002830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 422/842] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:14:59.003956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.004238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.004511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.022222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:59.023621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:59.046953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a23048-8b95-4451-9a54-b6c803fc3460 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.081518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.089096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.090181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.090627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.091362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.097957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.105328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.107568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.115368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 423/843] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:14:59.125969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:59.128186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:14:59.129237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:14:59.129237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:14:59.129237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:59.129891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 421/844] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:59.136386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.141143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.258930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae52bc-96ad-46f4-913c-9eb6cb2fc47a None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.268031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f0d06-e0e1-4d64-9625-6e93936d59f4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.300170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.300915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.301202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.301465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.302072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.303602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.309975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.310845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.311363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.311860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.312697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:59.313201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:59.313940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:59.314908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:59.315786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 424/845] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:59.322875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.323738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.339488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 422/846] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:59.342073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.381056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.393037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.437184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2208e5-6c0b-4e75-967e-969325773df6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.454465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672b2cf-f9ab-4ff5-8100-79f060b60f14 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.478244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.479041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.479351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.479633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.480254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:59.480558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.481946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d868f-d2ac-4101-8f20-cb5340ac4ed0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.486417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 425/847] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:59.487583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a949eb6-a3a5-4904-8338-5ae3ddc78666 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.487863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a949eb6-a3a5-4904-8338-5ae3ddc78666 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.488131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a949eb6-a3a5-4904-8338-5ae3ddc78666 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.493142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 426/848] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:14:59.493142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a949eb6-a3a5-4904-8338-5ae3ddc78666 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.499923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.500838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.501359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.502273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.512276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.514210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-6a0c97d0-e7da-4d50-b413-d40ab0e6a749 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:14:59.526926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:59.527902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:14:59.528823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 423/849] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:14:59.529312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.529906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.529906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.551106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:14:59.552318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:14:59.607338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a949eb6-a3a5-4904-8338-5ae3ddc78666 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.621225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.622291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 424/850] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:14:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:14:59.623336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.662617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.663188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.663687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.664140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.664567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.665334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:59.665831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.667383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.680006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 427/851] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:59.682148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.758350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0225e-ac8e-41f7-bcda-ebc6055e0716 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.800053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.800949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.801405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.801868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.802763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:14:59.803256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:14:59.805183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:14:59.825639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e9350-8ed4-4704-a687-86e908a5b109 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:14:59.831759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 425/852] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:14:59.833103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.869833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:14:59.870997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:14:59.871829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:14:59.872721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:14:59.894663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:14:59.899903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:14:59.900989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:14:59.901850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 428/853] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:14:59.906809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:14:59.949088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.963701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:14:59.972957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055416c-fcc3-435e-aefa-9f2a7bac4fa5 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.011184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.012041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.012515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.012787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.013634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=aaf8d31c016449e6aa6408c7f69abe20) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:00.014117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.015577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.022078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 426/854] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/role_assignments?user.id=aaf8d31c016449e6aa6408c7f69abe20&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:00.035621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f1a78-c5a1-45e7-b094-818376b31cd7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.077250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.077901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.078182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.078459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.079178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:00.079519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.080955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.091582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 429/855] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:14:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:00.096943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.209137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2712a04-d50f-4709-a05c-40ad18a62b3a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.244537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749b58e-392c-44da-a089-94534b672eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.245176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b58e-392c-44da-a089-94534b672eab None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.245447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b58e-392c-44da-a089-94534b672eab None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.245713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749b58e-392c-44da-a089-94534b672eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.256499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b58e-392c-44da-a089-94534b672eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.257954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749b58e-392c-44da-a089-94534b672eab None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-9749b58e-392c-44da-a089-94534b672eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:00.274813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:00.276078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:00.277207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:00.277207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:00.277207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:00.277207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 427/856] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:00.283279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.291926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927bb3a-ecb4-4ba4-9e6e-8074804efb47 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.292455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927bb3a-ecb4-4ba4-9e6e-8074804efb47 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.292921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927bb3a-ecb4-4ba4-9e6e-8074804efb47 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.294692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 428/857] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:00.299966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.300482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.300930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.319690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:00.321129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:00.323911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:00.389615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc6e52-fb24-4568-874f-78f31fd523d3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.395578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 429/858] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:00.403571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:00.414649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.425912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.426444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.426722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.426999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.427751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:00.428061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.429505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.446899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 430/859] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:00.448770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.542205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974fa7cf-ce9d-4bdc-8553-97aeba0c1df2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.571126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf626c-d427-4895-8c70-1041de60d541 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.581206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.582149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.582567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.583081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.583921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.585434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:00.595008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:00.596143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:00.597235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:00.604796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 430/860] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:00.606384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.609360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.609915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:00.610063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.610443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.610785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.618827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.621233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:00.640542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:00.641509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:00.642540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:00.650708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 431/861] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:00.653906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.689697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:00.742275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69120de0-81d3-4a9b-9162-d35b4329f2b6 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.779424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.780032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.780305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.780614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.781298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:00.781528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.782882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.792891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 431/862] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:00.794155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.810345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32261d2-afd8-463d-87fb-20d60f78bed3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.859179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.860418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.861256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.862004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.863129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:00.863853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.866454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.897369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 432/863] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:00.897539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:00.898654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:00.907767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afce21-52ed-4d2f-b6a4-fbf795a9d99c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:00.944971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:00.945623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:00.951471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:00.951471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:00.951471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:00.951471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:00.951471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:00.959373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 432/864] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:00.997602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:01.024087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5b0e3-bc84-4772-93e3-3603f9a0ffba None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:01.064307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:01.064933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.065212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.065487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.140547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:01.142037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:01.222176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:01.250197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 433/865] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:00 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:01.255879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:01.360659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97114235-d631-4fd4-bb20-44114743cdc7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:01.398513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:01.402914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:01.403935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.404461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.404928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.415911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:01.417391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:01.432482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:01.433483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:01.435138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:01.435138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:01.435138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:01.435138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 433/866] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:01.439806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:01.544943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e334329-3634-4a35-bc0a-b7b3efed64e3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:01.581858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:01.582465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.582746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.583022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.583697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:01.584004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:01.585382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:01.597958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 434/867] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:01.603615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:01.711695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92e241-c8cc-4d22-a892-0f9f4194cb9c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:01.748843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:01.749478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.749760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.750046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.757224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:01.758675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-f261e98f-5017-4ecf-88b6-589af95f2108 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:01.770791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:01.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:01.772630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 434/868] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:01.780340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:01.818263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:01.825978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a520f4-c0fe-4381-8abb-13e30688320f None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.826269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a520f4-c0fe-4381-8abb-13e30688320f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.826547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a520f4-c0fe-4381-8abb-13e30688320f None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.828162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 435/869] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:01.833012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.839411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.839411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.852559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:01.854152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:01.892332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f42382-7189-4a52-91c8-8c2a4888660b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:01.898382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:01.932439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:01.933156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:01.933434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:01.933705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:01.934393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:01.934697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:01.936578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:01.940863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 436/870] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:01.963456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 435/871] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:01.967736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:01.969739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:02.068246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:02.075066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42efffd-740a-4641-ab4a-8f151e1ae85a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:02.104803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7037ed-45bf-4b46-b5fb-8039e2a7f20a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:02.112450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:02.113153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.113497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.113888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.114685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=91ca9da53efa4421a21d5a824e25a483) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:02.115049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:02.116552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b14bc-50ce-49f9-a82e-28f3c1c3ef5b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:02.131438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 436/872] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:01 2022] GET /identity/v3/role_assignments?user.id=91ca9da53efa4421a21d5a824e25a483&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:02.144987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:02.145615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.145893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.146167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.146840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:02.148356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:02.195579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f9f2-cd53-45d0-b85e-5dc7dbdeb598 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.195935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f9f2-cd53-45d0-b85e-5dc7dbdeb598 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.196124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698f9f2-cd53-45d0-b85e-5dc7dbdeb598 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.198293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 437/873] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:02.204157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.205859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.205859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.226057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:02.227570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:02.240023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 437/874] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:01 2022] POST /identity/v3/services => generated 251 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:02.324288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 438/875] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:02 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:02.347166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:02.383200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:02.465107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1d228-cc6f-467d-8b21-78e29ad3082b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:02.500405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:02.502054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.502054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.502054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.502328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:02.503482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:02.512945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:02.513988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:02.514874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 438/876] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:02.522265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:02.613268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:02.629415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cf67f-cae3-4f37-b3cd-f7f3f491c853 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:02.665304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:02.665972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.666253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.666532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.667166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:02.667500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:02.668869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1b782-c8ab-4aa9-a4f0-173ac8cfbd12 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:02.674160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 439/877] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:02.679646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:02.784770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:02.791019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88232ada-ad8b-4cc2-a371-4eb806b66c03 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:02.828161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:02.828769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:02.829047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:02.829319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:02.829947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:02.830251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:02.831680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a496e-c3bd-4f6a-a6ae-31ec3c3ad9d0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:02.838825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 439/878] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:03.124216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6922f0-ffdb-4803-a08f-5080f67f6f1d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:03.124733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6922f0-ffdb-4803-a08f-5080f67f6f1d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:03.124809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6922f0-ffdb-4803-a08f-5080f67f6f1d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:03.126397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 440/879] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:03.131571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b3d3e-45c3-439b-ad96-4544c668f751 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:03.131858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b3d3e-45c3-439b-ad96-4544c668f751 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:03.132133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b3d3e-45c3-439b-ad96-4544c668f751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:03.150825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0a8b3d3e-45c3-439b-ad96-4544c668f751 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:03.152042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0a8b3d3e-45c3-439b-ad96-4544c668f751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:03.224870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 440/880] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:03.248312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6922f0-ffdb-4803-a08f-5080f67f6f1d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:03.351459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6922f0-ffdb-4803-a08f-5080f67f6f1d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:03.387295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426e639-63b1-4efb-b841-9baefcbb4fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:03.387863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e639-63b1-4efb-b841-9baefcbb4fd7 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:03.388126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e639-63b1-4efb-b841-9baefcbb4fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:03.388404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426e639-63b1-4efb-b841-9baefcbb4fd7 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:03.389067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e639-63b1-4efb-b841-9baefcbb4fd7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:03.390447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426e639-63b1-4efb-b841-9baefcbb4fd7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:03.497640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 441/881] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:03 2022] POST /identity/v3/services => generated 269 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:04.385733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351f333-f15a-467f-b8db-b2d213636dc0 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.386619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351f333-f15a-467f-b8db-b2d213636dc0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.387042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351f333-f15a-467f-b8db-b2d213636dc0 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.388791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 441/882] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:04.394307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.395145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.395673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.414552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:04.416112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:04.419256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:04.425349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b042b7-8ab8-488c-88c0-d3b067093eb0 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.425349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b042b7-8ab8-488c-88c0-d3b067093eb0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.425349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b042b7-8ab8-488c-88c0-d3b067093eb0 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.425349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 442/883] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:04.430306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.430580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.430844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.452506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:04.453704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:04.510713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 442/884] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:04.529718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:04.530321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 443/885] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:04.553428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:04.577366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:04.642436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf1f42-92e6-4675-b793-ac66b2090f44 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:04.679695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3512eb-151e-44e5-a8a3-ac98b478e47b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:04.684174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:04.685227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.685713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.686185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.699363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:04.704782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:04.705692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:04.706550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:04.709188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:04.715283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 443/886] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 00:15:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:04.716381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:04.718662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:04.719560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:04.737294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:04.737883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:04.738793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:04.740007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 444/887] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:15:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:04.742109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:04.824176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2951a30-34f8-49c2-a41b-d1fe8c47e31f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:04.859531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:04.860633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:04.861420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.861868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.862293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.863127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:04.863688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:04.865217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:04.874102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 444/888] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:15:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:04.878841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a28e3-39dd-4f71-a3a6-2fae3483a8bd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:04.880292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:04.924625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:04.925546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:04.925829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:04.926140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:04.926728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:04.927445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:04.929119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:04.940839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 445/889] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:15:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:04.942494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:04.951327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.003110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e61be-1574-412f-95e0-21139fb7b78c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.038082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.038670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.038946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.039216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.047097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839c378-bd66-44bc-a284-ca503a0f6cc4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.049942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.051157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a7c4d6fc-92e1-4972-9d16-f858ff6d93c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:05.066217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:05.067481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:05.068377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:05.068377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:05.068377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:05.073031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 445/890] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:05.074029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ea23-6553-48f4-a766-b87b02a755ec None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.074329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ea23-6553-48f4-a766-b87b02a755ec None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.074599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ea23-6553-48f4-a766-b87b02a755ec None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.079293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 446/891] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:05.079293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ea23-6553-48f4-a766-b87b02a755ec None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.088203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.088946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.089220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.089497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.090124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:05.090422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.091816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769c4f6-cac5-4410-8221-466b59e87702 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.096877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 446/892] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:15:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:05.098065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.098349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.098622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.120602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:05.122217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:05.132407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.183814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ea23-6553-48f4-a766-b87b02a755ec None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.196973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.215276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 447/893] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:05.215276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.220544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.221213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.221425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.221754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.222388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:05.222683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.224117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.237215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 447/894] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:05.237959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.331076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6da3-d3db-442c-aa88-0737a64fa03c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.346892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fee7b-c4c2-4da3-83b5-410445a7b89a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.366061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.366668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.366948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.367256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.367943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] RBAC: Adding query filter params (interface=public, service_id=3848132d588b4494b5fa05c1f099003d, region_id=RegionOne) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:05.368250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.370335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.380292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 448/895] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/endpoints?service_id=3848132d588b4494b5fa05c1f099003d&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:05.380292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.388299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.390418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.391415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.391694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.391962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.392630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.393995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.483340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 448/896] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:05 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:05.483340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.491333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f57979-bbe4-421a-ad59-a6639d217443 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.520287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.550944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.552538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.552830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.553102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.564932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.567665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:05.583806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:05.584863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:05.586105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 449/897] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:05.592325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.605618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30527b0-885f-486b-ba74-4fd7ff3b988c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.645819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853da396-e8e0-40e0-866f-f1e37dba4b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.652968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853da396-e8e0-40e0-866f-f1e37dba4b42 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.653273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853da396-e8e0-40e0-866f-f1e37dba4b42 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.653549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853da396-e8e0-40e0-866f-f1e37dba4b42 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.654200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853da396-e8e0-40e0-866f-f1e37dba4b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.655656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853da396-e8e0-40e0-866f-f1e37dba4b42 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.662933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 449/898] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:15:05.667496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d41fc-eb1f-4cb7-aa0f-504f91a413ec None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.667496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d41fc-eb1f-4cb7-aa0f-504f91a413ec None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.667496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d41fc-eb1f-4cb7-aa0f-504f91a413ec None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.667496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 450/899] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:05.673302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.673612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.673896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.678447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.694419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:05.695710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:05.739662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702fb9bc-e686-470d-8998-f98484d84cdd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.789742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.791129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.791774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.792271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.793075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:05.793449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.795051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:05.796032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 451/900] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:05.820947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.826090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 450/901] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:05.832915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:05.857858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:05.944979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986e01d-646b-4d62-bc0b-34567e402c9b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.980874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee769-461e-4af9-886e-6a339cc28656 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:05.990901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:05.991869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:05.992251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:05.992621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:05.993332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:05.994974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:06.004614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:06.005805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:06.007734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:06.010550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.015194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 452/902] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:06.015194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.022397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.023094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.023474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.023822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.112160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.113365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.114829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.131148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be867b64-823a-4094-bbfe-32031f573ba3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.176674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.177284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.179407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.179693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.180331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:06.180637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.182119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.192657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 453/903] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:06.194140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.237758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 451/904] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:05 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:06.243317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.267778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.315580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037ca2-c516-42fd-9e86-fd70874e1136 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.359592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.360209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.360490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.360766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.361402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:06.361521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b65bf-78f7-4176-be8a-0785626b68f9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.361691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.363065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.370372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 454/905] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:06.375722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.395316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.412200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.414163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.414601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.415040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.430867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.432592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:06.448306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:06.449299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:06.450182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:06.450182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:06.450182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:06.454288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 452/906] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:06.455126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.492650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e9535-b5bf-4907-b8eb-675d7998c72f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.529283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.529807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.530451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.530451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.531038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] RBAC: Adding query filter params (interface=public, service_id=117fdcecab5c4b71aea0c47205fa67c1, region_id=RegionOne) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:06.531397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.532872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.543533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 455/907] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/endpoints?service_id=117fdcecab5c4b71aea0c47205fa67c1&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:06.544376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.551898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.586359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf77d7e-4f60-4444-9410-67fc86e36a09 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.632442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.633446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.633961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.635752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.637261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:06.638024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.639420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.641315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.658779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 453/908] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:06.667343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.673146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31605380-d80c-4687-8a51-6ad7f61958cf None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.710027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.711032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.711551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.712038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.713406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.715160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.721937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 456/909] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:15:06.784359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fd717-16c0-4890-97af-f74aaabe6e65 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:06.819458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:06.824108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:06.824754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:06.825030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:06.825299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:06.833330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:06.834808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:06.848114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:06.849093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:06.850422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 454/910] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:06.857393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:06.968508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7834-3a5a-4491-abdb-348ddc28840f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:07.003319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:07.004359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.004834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.005313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.006250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:07.006774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:07.008421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e7dcd-a95f-4695-8fc6-bd449203eaff None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:07.025166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 457/911] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:07.028585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:07.131134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afa102-fa9b-4e4e-8e0c-b86c54177dc5 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:07.165853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:07.166880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.167408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.167901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.168829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] RBAC: Adding query filter params (role.id=a0e5c3773fb54d60a95f39cde0a6a625, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=91ca9da53efa4421a21d5a824e25a483) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:07.169334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:07.170987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f2b9c-279c-4717-bb4c-70fd6044313a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:07.181673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 455/912] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:07 2022] GET /identity/v3/role_assignments?user.id=91ca9da53efa4421a21d5a824e25a483&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=a0e5c3773fb54d60a95f39cde0a6a625 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:07.671119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bad3-df0b-4825-9295-47139ad5752d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.673544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bad3-df0b-4825-9295-47139ad5752d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.674939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bad3-df0b-4825-9295-47139ad5752d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.676959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 458/913] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:07.682752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28611-8ffe-4800-ac1d-5f737315720c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.683167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28611-8ffe-4800-ac1d-5f737315720c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.683391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f28611-8ffe-4800-ac1d-5f737315720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.701697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-10f28611-8ffe-4800-ac1d-5f737315720c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:07.702869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-10f28611-8ffe-4800-ac1d-5f737315720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:07.754380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:07.770913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 456/914] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:07.794926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6bad3-df0b-4825-9295-47139ad5752d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:07.799710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:07.881290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5cb0f-d59d-493b-9e53-e4787a103600 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.881290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5cb0f-d59d-493b-9e53-e4787a103600 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.881290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5cb0f-d59d-493b-9e53-e4787a103600 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.881935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 457/915] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:07.887839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.888363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.888851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.905597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6bad3-df0b-4825-9295-47139ad5752d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:07.908393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:07.909789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:07.950701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:07.951706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:07.952208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:07.952694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:07.953479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:07.955152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:07.965542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:07.966566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:07.967457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:07.970279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 459/916] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:07.979275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:07.991340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 458/917] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:08.008634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.062119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:08.104214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc732f-3b28-4114-90a0-2fc879e6fbc9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.129468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a400a-9506-430d-a58b-55ff81f64428 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.148429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.149357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:08.149474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.151173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.151481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.152112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:08.152766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.154134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.163470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 460/918] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:08.167359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.176762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.177438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.177739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.178304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.178611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.180054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:08.190148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:08.191165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 00:15:08.192335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:08.197430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 459/919] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:15:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:08.197871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.282914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56baabfd-e04e-44c3-80b5-7c20125f6508 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.306374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2d2e-a98a-449d-9918-ab650340465e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.316279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:08.324089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.329306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.329306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.329306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.329306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:08.329306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.329912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.336705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 461/920] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:08.343409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.350582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.351189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.351495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.351765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.352390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:08.352687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.354074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.363997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 460/921] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:15:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:08.364868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.416298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:08.459379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbd8e0-a781-4855-92cb-ff529a736339 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.478214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569282de-3637-46aa-b3f0-26149aff949a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.496742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.497702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.498196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.498681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.499617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] RBAC: Adding query filter params (interface=public, service_id=5839e42d86fd41f393359614b537276e, region_id=RegionOne) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:08.500146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.501728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.507604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 462/922] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:15:08 2022] GET /identity/v3/endpoints?service_id=5839e42d86fd41f393359614b537276e&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:08.513377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.522712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.523363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.523643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.523916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.524596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:08.524984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.526871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.537677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 461/923] 10.222.0.15 () {56 vars in 1114 bytes} [Mon Aug 8 00:15:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:08.538997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:08.586296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:08.629273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504bb2d-305f-4ec3-acc5-fd54c8bea2ff None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.657792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0af7d0-9d49-4451-be23-3adc6f1a4250 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:08.664941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3063017-e761-4171-854e-94f0c1aa2d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.665864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3063017-e761-4171-854e-94f0c1aa2d04 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.666146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3063017-e761-4171-854e-94f0c1aa2d04 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.666420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3063017-e761-4171-854e-94f0c1aa2d04 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.667165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3063017-e761-4171-854e-94f0c1aa2d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.668416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3063017-e761-4171-854e-94f0c1aa2d04 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.669495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:08.675268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 463/924] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:15:08 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:15:08.704391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7d312-0074-4ce2-9586-e73639bd9cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:08.705788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d312-0074-4ce2-9586-e73639bd9cb2 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.706272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d312-0074-4ce2-9586-e73639bd9cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.706749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7d312-0074-4ce2-9586-e73639bd9cb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.707784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d312-0074-4ce2-9586-e73639bd9cb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:08.709485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7d312-0074-4ce2-9586-e73639bd9cb2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:08.802163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 462/925] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:15:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:08.962736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee031308-9b87-4883-a81c-170e7a7bc1a4 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.963100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee031308-9b87-4883-a81c-170e7a7bc1a4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.963358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee031308-9b87-4883-a81c-170e7a7bc1a4 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.964883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 464/926] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:08.969951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c25abd-ae99-49fc-a2c0-f528081252f3 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:08.970238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c25abd-ae99-49fc-a2c0-f528081252f3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:08.970596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c25abd-ae99-49fc-a2c0-f528081252f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:08.991264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d7c25abd-ae99-49fc-a2c0-f528081252f3 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:08.992841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d7c25abd-ae99-49fc-a2c0-f528081252f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:09.073140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 463/927] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:09.095292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee031308-9b87-4883-a81c-170e7a7bc1a4 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.207094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee031308-9b87-4883-a81c-170e7a7bc1a4 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.232860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:09.240239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a888b-16be-4d96-952e-806da2b64bc0 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.240533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a888b-16be-4d96-952e-806da2b64bc0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.240803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a888b-16be-4d96-952e-806da2b64bc0 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.244953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.245898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.246366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.246666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 464/928] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:09.247033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.247884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:09.248204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.248489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.248766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.250125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:09.260019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:09.261109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 00:15:09.262038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:09.264536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 465/929] 10.222.0.15 () {56 vars in 1132 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:09.269496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:09.269496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:09.271601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.353877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 465/930] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:09.376079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.398047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ddc83-a3bb-4c7d-b84d-1c6f19677276 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.401809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:09.445588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.446469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.446947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.447353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.448337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:09.448647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:09.450587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:09.461429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 466/931] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:09.462877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.500647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f317c0-e9e6-412c-a7dc-bccd6c8eb869 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.532376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:09.536481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.537327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.537796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.538268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.549780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:09.554937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:09.561195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:09.562061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 466/932] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:09.568698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.580601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afa517-e688-4859-8b09-ffc82aecee38 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.628086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.629522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.630437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.630930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.632109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:09.632858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:09.635751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:09.650612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 467/933] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:09.651863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.684043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:09.705335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04208a5b-5162-4c39-a80d-b93baa033dad None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.744880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.745483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.745807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.746075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.746788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:09.747090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:09.748500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:09.757412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 467/934] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:09.763420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:09.783344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741827f1-7879-42d5-bdce-eeac7b3d5df9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.805912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:09.828891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.829747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.830029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.830301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.831162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:09.832948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:09.894620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9ee89-8938-4438-8679-90ff29a39005 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:09.929068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:09.929689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:09.929970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:09.930243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:09.941544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:09.943263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:09.945057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:09.956988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 468/935] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:15:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:09.958554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:09.959620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:09.960540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:09.960540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:09.960540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:09.964321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 468/936] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:09.965658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.094889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307669a-cce3-4283-ad6e-e9be82aa7479 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:10.132855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:10.133466 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.133745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.134014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.135522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:10.135880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:10.137300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:10.149549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 469/937] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:10.155626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.266258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ed1f4-df30-42f8-ac97-04e3fb122cb3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:10.300766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:10.301719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.302192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.302638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.311639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:10.313293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:10.325655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:10.326651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:10.327648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:10.334247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 469/938] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:15:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:10.335904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.365951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:10.440975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c126b2-7ad7-4e5d-a996-923c7eeecaf3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:10.483169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:10.483806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.484081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.484353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.485066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:10.485365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:10.486787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:10.505519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 470/939] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 00:15:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:10.511919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.571507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:10.619466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a35a014-cc19-4256-8218-2e42b184702e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:10.655777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:10.656410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.659255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.659255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.668975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:10.669850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-e62a6e80-9133-448d-9278-d18b25e6edc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:10.685090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:10.686096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:10.687468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:10.687468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:10.687468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:10.687468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 470/940] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:10.693934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.783283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:10.799385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b112d-ad7f-486a-b7e4-cf65eaf0b3ec None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:10.831124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04e76-5fe1-4dd9-9618-67321e98c5b1 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.831685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04e76-5fe1-4dd9-9618-67321e98c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.831685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04e76-5fe1-4dd9-9618-67321e98c5b1 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.833284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 471/941] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:10.838689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.839219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.839717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.839840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:10.840425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:10.840939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:10.841395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:10.842110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:10.842485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:10.843955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:10.856889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 471/942] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:10.862534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:10.863755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:10.864430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.943250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 472/943] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:10.961723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:10.975035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0aca-4801-4f51-bed6-a4a3f352acb0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.007843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.013182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.013819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.014095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.014415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.023524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.025555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f370bd18-4de8-4c26-9290-43a30379cc74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:11.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:11.037991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.039316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 472/944] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:11.040153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb318c-8733-4f19-9527-e691d43d5dfc None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.040153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb318c-8733-4f19-9527-e691d43d5dfc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.040473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb318c-8733-4f19-9527-e691d43d5dfc None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.042467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 473/945] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:11.047130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb318c-8733-4f19-9527-e691d43d5dfc None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.057319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.084066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe3594-e848-40df-97bb-fb16046f52c2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.118828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.119474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.119754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.120026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.120607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.121970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-fd7d1554-b59b-4ccc-95fc-bb2ceb97ca3b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:11.131462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:11.132590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:11.133486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 473/946] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:11.135095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.135095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.135095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.152712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:11.153889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:11.164941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb318c-8733-4f19-9527-e691d43d5dfc None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.201925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.202842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.203349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.203696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.204472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:11.204783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.206219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.226937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 474/947] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:11.226937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.229025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 474/948] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:11.229796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.271016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.303056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.340301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632107d4-b1f1-40b2-b7ea-c1d7940db998 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.344196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c90254-a8e7-4880-9e30-2d24762e8b81 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.374636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.375256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.375537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.375802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.376469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:11.376771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.378112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.381181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.381832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.382109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.382379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/91ca9da53efa4421a21d5a824e25a483/roles/a0e5c3773fb54d60a95f39cde0a6a625` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.383081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 475/949] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:11.384833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=91ca9da53efa4421a21d5a824e25a483, role_id=a0e5c3773fb54d60a95f39cde0a6a625)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.386280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.388364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.463488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 475/950] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:15:11 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/91ca9da53efa4421a21d5a824e25a483/roles/a0e5c3773fb54d60a95f39cde0a6a625 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:15:11.464757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.505580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6ab26-e1b9-47d0-978e-fb8de3fb5350 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.542499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.550052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.550680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.550960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.551597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.552216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.558862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.558862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING keystone.server.flask.common [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 00:15:11.593974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.604732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a89d-b7a6-4a6c-a817-4694b7db1812 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.623262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.sql.core [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 00:15:11.623262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 00:15:11.623262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [parameters: {'id': '17dde81fbb274d7293818934922d77f9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 00:15:11.623262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91988) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17dde81fbb274d7293818934922d77f9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17dde81fbb274d7293818934922d77f9. Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application conn.query(q) Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 00:15:11.635742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result.read() Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 00:15:11.636651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application next(self.gen) Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application yield resource Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application next(self.gen) Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise self.value Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application session.commit() Aug 08 00:15:11.637596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.session.flush() Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application compat.raise_( Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 00:15:11.638789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 00:15:11.639773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application conn.query(q) Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application result.read() Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 00:15:11.642396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [parameters: {'id': '17dde81fbb274d7293818934922d77f9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.647723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:11.648661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17dde81fbb274d7293818934922d77f9. Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.650559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 476/951] 10.222.0.15 () {60 vars in 1140 bytes} [Mon Aug 8 00:15:11 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:11.654542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.658225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.659986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.659986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.659986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.660595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:11.660957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.662593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.674081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 476/952] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:11.680088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.766124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57cc4e1-9978-4be8-b2b7-97813523f6cc None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.788040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fdfac6-c724-4f9d-9ce4-98135f6cb482 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:11.801915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.802642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.802989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.803355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.810418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.812174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:11.823931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:11.825058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:11.826233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:11.831255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 477/953] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:11.833001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:11.835042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfef9e-95c3-4ce5-8e22-cdebacf85b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:11.835692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfef9e-95c3-4ce5-8e22-cdebacf85b67 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:11.835967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfef9e-95c3-4ce5-8e22-cdebacf85b67 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:11.836233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfef9e-95c3-4ce5-8e22-cdebacf85b67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:11.836886 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfef9e-95c3-4ce5-8e22-cdebacf85b67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:11.840153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfef9e-95c3-4ce5-8e22-cdebacf85b67 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:11.889221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:11.925886 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 477/954] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:15:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:11.955926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fea160-74c4-4957-adbd-a80c870c7770 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:12.005838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:12.006870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:12.007405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:12.007932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:12.008916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:12.010279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:12.012235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ab4a2-23e7-4ecc-a95a-3301d6f8ddf5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:12.033433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 478/955] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:11 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:12.148318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8abc0c-4148-4280-8d32-99244e20210e None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:12.148598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8abc0c-4148-4280-8d32-99244e20210e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:12.148873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8abc0c-4148-4280-8d32-99244e20210e None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:12.151092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 478/956] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:12.156847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:12.157126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:12.157401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:12.177635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:12.179185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:12.260558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 479/957] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:12.282740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8abc0c-4148-4280-8d32-99244e20210e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:12.301333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:12.397355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8abc0c-4148-4280-8d32-99244e20210e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:12.435981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:12.436617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:12.436892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:12.437162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:12.437803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:12.439252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:12.449361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:12.450435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:12.451896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 479/958] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:12.456271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:12.501313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:12.577199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd040605-4086-437f-9ed7-249bf7150c1c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:12.613748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:12.614735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:12.615252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:12.615699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:12.616516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] RBAC: Adding query filter params (name=compute) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:12.617007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:12.619151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20dcc6-31f1-4edc-aff3-c70a0e3dd83d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:12.625225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 480/959] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:12.631277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:12.718006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:12.745530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84afca87-9f67-4239-977f-dcbd077b59a1 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:12.780675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:12.781595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:12.782095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:12.782541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:12.783430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] RBAC: Adding query filter params (type=compute) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:12.783925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:12.785457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094101a4-4c07-4c3d-8bf2-0961fd02acbe None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:12.790978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 480/960] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:13.583408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5670a-2871-4013-a8e0-a53a885b4568 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:13.583970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5670a-2871-4013-a8e0-a53a885b4568 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:13.583970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5670a-2871-4013-a8e0-a53a885b4568 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:13.585533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 481/961] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:13.591213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:13.591639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:13.591963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:13.609954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:13.611131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:13.686274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 481/962] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:13.709251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5670a-2871-4013-a8e0-a53a885b4568 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:13.812971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:13.820772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5670a-2871-4013-a8e0-a53a885b4568 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:13.857863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98b3d6-823e-4672-ac73-33e8c544862a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:13.858549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98b3d6-823e-4672-ac73-33e8c544862a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:13.858831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98b3d6-823e-4672-ac73-33e8c544862a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:13.859103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98b3d6-823e-4672-ac73-33e8c544862a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:13.870417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98b3d6-823e-4672-ac73-33e8c544862a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-9d98b3d6-823e-4672-ac73-33e8c544862a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:13.875081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:13.876065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:13.877415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 482/963] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 00:15:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:13.882212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:13.959165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:13.998326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f51be6-30a9-4259-a49b-f8367fefb4bb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.034081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.034684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.035047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.035347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.036007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:14.036305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.037663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.045895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f787db4-a5e5-443d-ba04-2d559b03ff6a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.046488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f787db4-a5e5-443d-ba04-2d559b03ff6a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.046930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 482/964] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:15:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:14.047306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f787db4-a5e5-443d-ba04-2d559b03ff6a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.051213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 483/965] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:14.052961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.054701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.055171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.055676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.075648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:14.077625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:14.167958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 484/966] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:14.174202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee4e672-efd6-4bfe-b5cb-0d4f122fca80 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.191594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.211795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.213512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.213999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.214550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.216796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:14.227518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.229515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:14.245285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:14.246295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:14.247211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:14.247211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:14.247211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:14.249510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 483/967] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:14.258374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.286931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:14.310305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb94a11-9823-4124-b195-1f790826adc9 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.354282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.354921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.355247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.355520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.356133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.357507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d2780053-ab58-483a-95e1-a1d6a85a835e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:14.366479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:14.367593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:14.373222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 485/968] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:14.374044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645c1a7-f8f4-4f44-a517-8ac5c08821cc None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.374044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645c1a7-f8f4-4f44-a517-8ac5c08821cc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.374044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645c1a7-f8f4-4f44-a517-8ac5c08821cc None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.374044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 486/969] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:14.377028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.377311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.377583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.379328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73611e13-5e91-4ff5-a7b8-288e14f867dc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.397296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:14.398487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:14.417832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.418978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.419509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.419952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.420743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:14.421189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.422831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.436686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 484/970] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:14.437852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.476651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 487/971] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:14.477864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.496444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:14.557150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90474f36-85c4-4568-92fa-b074bc5fc06a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.573278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:14.594017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.594561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913aed54-e28a-4728-8c18-bb398b2f9033 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.594907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.594983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.595289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.595857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:14.596154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.597673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.608141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 485/972] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:14.608141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.638936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.638936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.638936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.638936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.652090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.653554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:14.666743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:14.667796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 00:15:14.668769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:14.673474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 488/973] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:14.673773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.715070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645efe-55d9-44b1-9846-6660ba710371 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.723141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:14.753275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.753883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.754163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.754437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.755186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.756763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.759902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING keystone.server.flask.common [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 00:15:14.779310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a25bb-565a-4943-9034-93f7e9c9cdc8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.799844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.sql.core [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 00:15:14.799844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 00:15:14.799844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [parameters: {'user_id': '4997ad10855f4951869e11038163b494', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 00:15:14.799844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91987) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 00:15:14.813209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.813824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.814366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.814479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.815058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:14.815408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application conn.query(q) Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application result.read() Aug 08 00:15:14.816393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 00:15:14.817712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application next(self.gen) Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application yield resource Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application next(self.gen) Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise self.value Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application session.commit() Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 00:15:14.823724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self.session.flush() Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application compat.raise_( Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 00:15:14.824830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 00:15:14.825906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application conn.query(q) Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application result.read() Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 00:15:14.826949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [parameters: {'user_id': '4997ad10855f4951869e11038163b494', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:14.828003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 00:15:14.828974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 00:15:14.830062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 00:15:14.830062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 00:15:14.830062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 00:15:14.830062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 00:15:14.830062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 00:15:14.830062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:14.830446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51457e-fdd5-46d3-9f0c-562d1dadcba7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.830539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 486/974] 10.222.0.15 () {60 vars in 1131 bytes} [Mon Aug 8 00:15:14 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:14.830539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.839131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 489/975] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:14.839131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c59a-1d8c-4ea9-ae20-a7bdac7d6c45 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.839131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c59a-1d8c-4ea9-ae20-a7bdac7d6c45 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.839131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c59a-1d8c-4ea9-ae20-a7bdac7d6c45 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.839131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 490/976] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:14.839131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c59a-1d8c-4ea9-ae20-a7bdac7d6c45 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.843134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:14.934125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698fd0f-fdf5-4265-9a31-74b18ab4d025 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.942412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c59a-1d8c-4ea9-ae20-a7bdac7d6c45 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:14.968115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.968715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.968989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.969259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.969975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:14.970279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.971949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:14.983858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:14.984501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:14.984781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:14.985054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:14.992614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:14.992900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 487/977] 10.222.0.15 () {56 vars in 1149 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:14.993759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:14.994139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-95478c0c-e9b4-4070-aa25-bc35ad74fee8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:15.007366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:15.008492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 00:15:15.010406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:15.011254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 491/978] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:15.012285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.012564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.012836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.026321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:15.030386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:15.031600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:15.091939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:15.097173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6db4a-4743-401e-b1a0-021b01c82592 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.100480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 492/979] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:15.101363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.130083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.130720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.130995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.131276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.131973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] RBAC: Adding query filter params (interface=admin, service_id=5839e42d86fd41f393359614b537276e, region_id=RegionOne) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:15.132273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.133634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.139252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 488/980] 10.222.0.15 () {56 vars in 1249 bytes} [Mon Aug 8 00:15:14 2022] GET /identity/v3/endpoints?service_id=5839e42d86fd41f393359614b537276e&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:15.139887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.205522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b419d-3bb2-41d3-b934-9939b4e92c48 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.233125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:15.238443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.239038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.239413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.239685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.242425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186a4926-be20-49f9-a7a5-f384e592076f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.276707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.277318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.277597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.277893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.278591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:15.278889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.280350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.299980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:15.307945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.309398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.316528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 489/981] 10.222.0.15 () {56 vars in 1101 bytes} [Mon Aug 8 00:15:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:15.317604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.425495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 493/982] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:15.425495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.459104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662f95d-9eec-4eec-95a2-2f5f66ee6d61 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.499760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.500656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.501140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.501564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.502360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.503775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.530935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:15.537920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ceb162-0a40-4d94-b721-816061c9971b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.572941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.573562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.573840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.574107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.574704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.576109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.581895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 494/983] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:15:15 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:15:15.582658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.602056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 490/984] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:15 2022] POST /identity/v3/services => generated 242 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:15.719174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4744f-a464-4364-8800-94230ac515f8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.768152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.768781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.769055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.769320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.781276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.782732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:15.798167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:15.799448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:15.800547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:15.800547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:15.800547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:15.800547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 495/985] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:15.805066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.817636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:15.909846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ca144-4115-47eb-950f-0367543c2d2f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:15.946927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:15.947417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:15.947693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:15.947963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:15.948660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:15.948955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:15.950312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:15.963338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 491/986] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:15.968005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:15.977529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:16.070790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6642147-156e-446b-8a1f-9c94b9bd772c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:16.106257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:16.106883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:16.107158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:16.107454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:16.114137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:16.115588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:16.128933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:16.129878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:16.130813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 496/987] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:16.136078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:16.206880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:16.241405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b75a80-0df4-4a78-b889-23fbc7a2cc61 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:16.276108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:16.276714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:16.276994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:16.277264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:16.277974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:16.278279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:16.279747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e306e4-f74a-4ab2-8b1b-9aec464eef2e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:16.297793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 492/988] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:16.299477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:16.367100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:16.402891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cf5f3-b911-4d73-a5cc-0c99c853a7b7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:16.436521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:16.437144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:16.437475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:16.437750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:16.438529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] RBAC: Adding query filter params (role.id=a0e5c3773fb54d60a95f39cde0a6a625, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=91ca9da53efa4421a21d5a824e25a483) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:16.438840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:16.440299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a266b6-450b-4e96-829b-c8b67316c309 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:16.447029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 497/989] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:16 2022] GET /identity/v3/role_assignments?user.id=91ca9da53efa4421a21d5a824e25a483&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=a0e5c3773fb54d60a95f39cde0a6a625 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:17.617809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026d02d-8b94-415f-afa2-df923a56278c None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:17.618396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026d02d-8b94-415f-afa2-df923a56278c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:17.618798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026d02d-8b94-415f-afa2-df923a56278c None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:17.620333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 493/990] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:17.625439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:17.625826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:17.625987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:17.644515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:17.645707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:17.672297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c84a-8a1d-4f27-89b9-4c29489548f0 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:17.672573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c84a-8a1d-4f27-89b9-4c29489548f0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:17.672845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c84a-8a1d-4f27-89b9-4c29489548f0 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:17.674308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 494/991] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:17.678931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:17.679207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:17.679510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:17.697263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:17.698755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:17.727272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 498/992] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:17.748809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:17.768246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 495/993] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:17.791484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:17.809035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:17.848091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:17.853572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec4b44-2235-4691-bade-3a662f4a76b2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:17.890277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:17.890467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a179a0-25a9-4df4-b2ae-8cdbab0cc13a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:17.890815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:17.891089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:17.891393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:17.892077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:17.893593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:17.903883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:17.904878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:17.906132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 499/994] 10.222.0.15 () {56 vars in 1126 bytes} [Mon Aug 8 00:15:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:17.910980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:17.928415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:17.929045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:17.929324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:17.929596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] PATH_INFO: `/v3/services/compute` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:17.930209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:17.931824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:17.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:17.941619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:17.942720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 496/995] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:17.948377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.018814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787b3c-2131-4197-8f58-5b51218d2268 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.043553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:18.049246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91852315-48af-465c-bda5-fee66b12a9bb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.059753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.060355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.060634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.060904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.061652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:18.061946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.063321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.068542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 500/996] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:18.074377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.075728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:18.087268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.087885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.088163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.088435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.089060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:18.089362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.090741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.095560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 497/997] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:18.100911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.188216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871ef4b-3e39-423d-a9cf-5fcfdda609c7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.209294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95e5bb-c54c-4fb8-bf0d-6868948d9002 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.223041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.223889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.224021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.224246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.224867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:18.225163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.226558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.232869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 501/998] 10.222.0.15 () {56 vars in 1122 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:18.238163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.246670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.247779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.248117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.248450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.249479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:18.249849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.251609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.256652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 498/999] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:18.262270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.297317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:18.317453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:18.348843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3335fc-ce1c-4e28-8886-197de6f23053 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.372535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4591-8ab0-4290-8bba-c40ab8b903e0 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.385628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe8ccdf-24ea-4f7b-9a98-7e0e9ad71c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.386365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ccdf-24ea-4f7b-9a98-7e0e9ad71c0f None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.386775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ccdf-24ea-4f7b-9a98-7e0e9ad71c0f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.387048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe8ccdf-24ea-4f7b-9a98-7e0e9ad71c0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.387735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ccdf-24ea-4f7b-9a98-7e0e9ad71c0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.389089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe8ccdf-24ea-4f7b-9a98-7e0e9ad71c0f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.412180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.412795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.413061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.413329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.414014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] RBAC: Adding query filter params (interface=public, service_id=5f867982a07a4b07b6ffbc9286859fa1, region_id=RegionOne) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:18.414318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.415767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.420937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 499/1000] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/endpoints?service_id=5f867982a07a4b07b6ffbc9286859fa1&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:18.426128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.463822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 502/1001] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:15:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:15:18.535997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ef815-313b-4df7-8a32-9f521bccb512 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.536334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ef815-313b-4df7-8a32-9f521bccb512 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.536561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ef815-313b-4df7-8a32-9f521bccb512 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.537946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 503/1002] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:18.543508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289f1d0-a178-4d67-98f9-f818f8c3f241 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.543735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.544044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.544416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.562095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:18.564108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:18.565362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:18.590649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.591289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.591566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.591830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.592461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.593899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.611259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 500/1003] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:15:18.656813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 504/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:18 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:18.679465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.789697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69f5f-c500-4be2-8d99-20122af77371 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.825032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.825660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.825935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.826201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] PATH_INFO: `/v3/services/image` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.826779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.828158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:18.836816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:18.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:18.839425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 501/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:18.846546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:18.891154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:18.948115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e5754-bfb4-4e8e-b73c-2403dd15e0ae None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:18.983949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:18.984563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:18.984841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:18.985110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:18.985731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] RBAC: Adding query filter params (name=image) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:18.986030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:18.987415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bd343-3893-4016-b658-97e71630f23c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:18.997452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 505/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:18.998633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:19.039260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:19.100466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e42440-e3fd-4bee-bc78-ccf97b7c3555 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:19.135488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:19.136096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:19.136375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:19.136647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:19.137273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] RBAC: Adding query filter params (type=image) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:19.137572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:19.138934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19f69-d70f-4012-a8e5-ad353ee09287 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:19.151518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 502/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:20.565303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbcaed-10a6-4806-a1fb-98b653723810 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:20.565826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbcaed-10a6-4806-a1fb-98b653723810 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:20.565904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbcaed-10a6-4806-a1fb-98b653723810 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:20.567505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 506/1008] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:20.573479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22355b45-6d50-4889-b379-0c324b349f17 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:20.573903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22355b45-6d50-4889-b379-0c324b349f17 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:20.574082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22355b45-6d50-4889-b379-0c324b349f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:20.592282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-22355b45-6d50-4889-b379-0c324b349f17 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:20.593484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-22355b45-6d50-4889-b379-0c324b349f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:20.665430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 503/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:20.672683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cbc9c-7f16-4c5c-b39d-b6ece19e6b85 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:20.672962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cbc9c-7f16-4c5c-b39d-b6ece19e6b85 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:20.673237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cbc9c-7f16-4c5c-b39d-b6ece19e6b85 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:20.674581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 507/1010] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:20.679921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051a132-e4be-4191-98dc-29df9694269f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:20.680213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051a132-e4be-4191-98dc-29df9694269f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:20.680492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051a132-e4be-4191-98dc-29df9694269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:20.694411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cbc9c-7f16-4c5c-b39d-b6ece19e6b85 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:20.702222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d051a132-e4be-4191-98dc-29df9694269f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:20.703529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d051a132-e4be-4191-98dc-29df9694269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:20.732461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:20.776509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 504/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:20 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:20.800834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051a132-e4be-4191-98dc-29df9694269f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:20.807010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cbc9c-7f16-4c5c-b39d-b6ece19e6b85 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:20.816193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:20.844228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:20.844960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:20.845241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:20.845506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:20.846604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:20.848276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:20.850690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 508/1012] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:20.856884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:20.910677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051a132-e4be-4191-98dc-29df9694269f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:20.946913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:20.947663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:20.947938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:20.948207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:20.948788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:20.950155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:20.958908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:20.959919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 00:15:20.960777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:20.961591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2f11e-186e-4227-96ce-9386fd7fd800 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:20.961679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 505/1013] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:20.966733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:20.995055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:21.000906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1348e6-25df-43bd-9fac-3bafe931cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:21.001531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1348e6-25df-43bd-9fac-3bafe931cba6 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.001822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1348e6-25df-43bd-9fac-3bafe931cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.002071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1348e6-25df-43bd-9fac-3bafe931cba6 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.002723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1348e6-25df-43bd-9fac-3bafe931cba6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:21.004169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1348e6-25df-43bd-9fac-3bafe931cba6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:21.068951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052661dc-0215-4ccc-a738-1e03232467ba None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:21.104478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:21.105145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.105428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.105695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.106309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:21.106605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:21.108032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:21.113026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 506/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:21.116578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 509/1015] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:20 2022] POST /identity/v3/projects => generated 311 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:15:21.121601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:21.156393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388719a-2682-4228-a49c-985e9af676ad None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.156994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388719a-2682-4228-a49c-985e9af676ad None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.157458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388719a-2682-4228-a49c-985e9af676ad None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.159032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 510/1016] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:21.164381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.164709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.164932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.183822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:21.185242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:21.208012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:21.229199 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:21.239186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17326a-6789-4356-9f5e-75bfc728a0d1 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:21.261942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 511/1017] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:21 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:21.279806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:21.280658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.280936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.281204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.281831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:21.282131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:21.283516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:21.286082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:21.291041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 507/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Mon Aug 8 00:15:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:21.296030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:21.406358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884af5b4-dfa9-4995-ba16-aeb49bcdbf7c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:21.413938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f91472-13b6-4947-9562-7845dbb20c65 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:21.443950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6cc6e-7548-4038-9fad-2fb5e41f1e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:21.444565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6cc6e-7548-4038-9fad-2fb5e41f1e00 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.444843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6cc6e-7548-4038-9fad-2fb5e41f1e00 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.445111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6cc6e-7548-4038-9fad-2fb5e41f1e00 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.445859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6cc6e-7548-4038-9fad-2fb5e41f1e00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:21.448050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6cc6e-7548-4038-9fad-2fb5e41f1e00 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:21.452401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8de19a-26fd-4f71-a494-68152cc2f65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:21.453246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de19a-26fd-4f71-a494-68152cc2f65a None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:21.453488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de19a-26fd-4f71-a494-68152cc2f65a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:21.453769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8de19a-26fd-4f71-a494-68152cc2f65a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:21.454429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de19a-26fd-4f71-a494-68152cc2f65a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:21.455838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8de19a-26fd-4f71-a494-68152cc2f65a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:21.566722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 512/1019] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:21 2022] POST /identity/v3/services => generated 242 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:15:21.602205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 508/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:15:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:15:23.237742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df903e6-bc06-42c2-93cc-361284d83131 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.238267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df903e6-bc06-42c2-93cc-361284d83131 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.238267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df903e6-bc06-42c2-93cc-361284d83131 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.239815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 513/1021] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:23.245681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.246084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.246291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.264573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:23.265752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:23.269513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:23.341423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 509/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:23.365959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df903e6-bc06-42c2-93cc-361284d83131 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:23.385050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:23.468681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df903e6-bc06-42c2-93cc-361284d83131 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:23.504164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71595819-e6dc-4686-8208-637213cb99c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:23.504695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595819-e6dc-4686-8208-637213cb99c9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.504972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595819-e6dc-4686-8208-637213cb99c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.505237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71595819-e6dc-4686-8208-637213cb99c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.506332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595819-e6dc-4686-8208-637213cb99c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:23.507774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71595819-e6dc-4686-8208-637213cb99c9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:23.509514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 514/1023] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:23.515107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:23.613324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb69bf1-f04f-4d9e-b1eb-ea973a5a349e None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.613669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb69bf1-f04f-4d9e-b1eb-ea973a5a349e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.613875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb69bf1-f04f-4d9e-b1eb-ea973a5a349e None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.614655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb230e-ef02-42a1-9fa0-9cd5855047e1 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:23.615361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 515/1024] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:23.620236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d858d-2e42-4ba2-b705-c5f9819f465e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.620518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d858d-2e42-4ba2-b705-c5f9819f465e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.620789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d858d-2e42-4ba2-b705-c5f9819f465e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.638517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-2e5d858d-2e42-4ba2-b705-c5f9819f465e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:23.639719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2e5d858d-2e42-4ba2-b705-c5f9819f465e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:23.654274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:23.654881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.655153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.655551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.656205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:23.657651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:23.668101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:23.716681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 516/1025] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:23.717932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9d7d9-7492-4dd0-b7d6-a5a268b46734 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.718212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9d7d9-7492-4dd0-b7d6-a5a268b46734 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.718480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9d7d9-7492-4dd0-b7d6-a5a268b46734 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.719762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 517/1026] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:23.724627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.724915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.725195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.744339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:23.745542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:23.756578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:23.778203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 510/1027] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:15:23 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:23.779025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:23.818396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 518/1028] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:23.842427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:23.897717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a511f3-a3c4-4b35-96b7-033a1b4674f0 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:23.940544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:23.941198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.941476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.941760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] PATH_INFO: `/v3/services/image` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:23.942642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:23.944053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:23.950640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa0845-bedb-4b45-b2a2-21c790be2efe None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:23.953558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:23.954538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:23.955499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 511/1029] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 8 00:15:23 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:23.961938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:23.995425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:23.996066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:23.996346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:23.996618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.010682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:24.015864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:24.018077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:24.019110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 519/1030] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 00:15:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:24.023553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.028237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.033166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.084243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77bc08b-9741-45d4-a306-d8243e40bd3a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.124615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.125223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.125507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.125788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.126418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] RBAC: Adding query filter params (name=image) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:24.126730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.128126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.133166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 512/1031] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:24.137985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94838b0-87eb-4fd3-ae9a-a2bb2499fe0d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.140156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.174484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.175091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.175402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.175677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.176335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:24.176639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.177993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.186362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 520/1032] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:24.192109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.263093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.266711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544f9ab-f6e1-45f0-b97a-afa5efa8c4a9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.359253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.374372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d9038-ed91-431a-9456-585d7e7e0525 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.375176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.375824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.376105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.376377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.377008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] RBAC: Adding query filter params (type=image) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:24.377313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.378689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.383681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 513/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:24.391179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.413537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.414174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.414456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.414727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.425816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.427272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:24.443983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:24.444951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:24.447460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:24.447460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:24.447460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:24.447460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 521/1034] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:24.451680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.504195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7c076-168b-4f71-b290-1465ad836d77 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.540156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.541317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.541592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.541857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.542738 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] RBAC: Adding query filter params (interface=public, service_id=1f134cf5098c45c986f5e94fd1a15a0a, region_id=RegionOne) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:24.543163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.544570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.555317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 514/1035] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/endpoints?service_id=1f134cf5098c45c986f5e94fd1a15a0a&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:24.556225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.558459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb295123-8d0d-4ddc-be6b-9fded54a042b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.576279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.594120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.597177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.597177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.597177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.597177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:24.597177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.597818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.609844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.619527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 522/1036] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:24.620446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.668282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b944e47-46dd-4f39-bab1-f5f27525a87e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.705609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.706334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.706512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.706760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.707387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.708804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.716653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 515/1037] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:15:24.741719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64555055-703f-4a02-af45-72d5693a49e7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.782743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.783425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.783706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.783976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.792898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.794288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:24.808059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:24.809085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:15:24.810071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:24.811785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 523/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:24.815040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:24.923559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d143498-e383-4751-9d6a-63cbce6c139f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:24.937214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:24.958548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:24.959274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:24.959546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:24.959814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:24.960567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:24.960873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:24.962438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:24.982273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 516/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:24.987817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:25.030721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:25.092072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d262af7-cc9b-4392-b15b-3548033132d6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:25.126121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:25.126723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.127000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.127294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.194478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:25.195950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:25.235129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:25.296311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 524/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 00:15:25.302002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:25.406825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49a802-5b67-4917-b911-8349fd2b4482 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:25.440753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:25.441382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.441826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.442251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.453400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:25.454841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:25.469999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:25.471064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:25.472593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:25.472593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:25.472593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:25.475921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 517/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:25.476902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:25.579659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786629a4-9355-4ac7-8991-8dd7adc3f7fe None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:25.620904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:25.621510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.621785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.622055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.622689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:25.622986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:25.624463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:25.631004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:25.638973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 525/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:25.644428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:25.683702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:25.762306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe341d0-809e-467b-9b23-a3a2f786f7b5 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:25.799797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:25.800325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.800600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.800868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.808673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:25.810137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-bff8262e-c1df-4b41-b869-52181b9f1e4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:25.822576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:25.823793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:25.824720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:25.827611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 518/1043] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:25.829751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:25.905724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffee84-000d-4009-b2a6-6c628618d83a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.906063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffee84-000d-4009-b2a6-6c628618d83a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.906272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffee84-000d-4009-b2a6-6c628618d83a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.908247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 519/1044] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:25.913017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.913296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.913567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.932583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:25.933701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:25.941764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c68fc6-714c-4bd4-b682-07dd0d7c13bc None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:25.978553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:25.979559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:25.979904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:25.980177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:25.980854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:25.981144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:25.982604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:25.995718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 526/1045] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:26.001448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.011295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 520/1046] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:26.035124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.049848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:26.067106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:26.110505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef3c97-a35e-45b3-99e8-3f6c35bd39ad None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:26.150045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44a220-5895-4808-b175-5a4fc2472a8d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:26.156052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:26.156660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.157312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.157588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.158319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] RBAC: Adding query filter params (role.id=a0e5c3773fb54d60a95f39cde0a6a625, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=c1031699d7da4a76965035faa9305dbd) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:26.158618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:26.160078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:26.171261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 527/1047] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/role_assignments?user.id=c1031699d7da4a76965035faa9305dbd&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=a0e5c3773fb54d60a95f39cde0a6a625 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:26.195146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:26.195761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.196042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.196840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.210061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:26.216031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:26.217021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:26.217881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 521/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:26.223493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.343984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ec080-102b-429e-9f0e-ce669c9bfbae None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:26.369429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:26.381675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:26.386108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.386108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.386108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.386108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:26.386108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:26.386462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:26.401020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 528/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:26.402321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.476513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:26.507965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4ddd9-e44c-46d7-b73f-64e6aea76906 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:26.544005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa78cc9-fb77-4179-8811-0e5e53702acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:26.544637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa78cc9-fb77-4179-8811-0e5e53702acf None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.544953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa78cc9-fb77-4179-8811-0e5e53702acf None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.545251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa78cc9-fb77-4179-8811-0e5e53702acf None admin] PATH_INFO: `/v3/users/4d5ee6e10323407785f5d83cce557bf2` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.558871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa78cc9-fb77-4179-8811-0e5e53702acf None admin] RBAC: Authorizing `identity:get_user(user_id=4d5ee6e10323407785f5d83cce557bf2)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:26.560363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa78cc9-fb77-4179-8811-0e5e53702acf None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:26.566678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 522/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/users/4d5ee6e10323407785f5d83cce557bf2 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:26.568153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.681833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e8b97-cae1-492a-8fc9-04c980cf8169 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:26.716049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:26.716683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.716957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.717223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.768265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:26.784140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:26.785587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:26.854017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b865-2173-4b6e-a785-7b4ecf5e516d None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.854296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b865-2173-4b6e-a785-7b4ecf5e516d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.854572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292b865-2173-4b6e-a785-7b4ecf5e516d None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.856086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 523/1051] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:26.860952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:26.861231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:26.861499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:26.879580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:26.880762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:26.894301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 529/1052] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 00:15:26.895725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.949466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 524/1053] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:26.971790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:26.991136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:27.007481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6270c3-df0f-4b3e-bb10-665d7956a00d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:27.035805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:27.047857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:27.048476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:27.049622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:27.049622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:27.050317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] RBAC: Authorizing `identity:get_project(project_id=e7fd08895d6f4f3294c034cc88721fe1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:27.051936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:27.053707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 530/1054] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:27.060209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:27.079241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4908e-583e-49f7-ad7e-9046ad074f69 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:27.114596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:27.115218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:27.115518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:27.115787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] PATH_INFO: `/v3/services/image` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:27.116361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:27.117730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:27.126999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:27.128183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:27.129116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 525/1055] 10.222.0.15 () {56 vars in 1105 bytes} [Mon Aug 8 00:15:26 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:27.133977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:27.171884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e1c73-e6f4-478b-acaa-0459c8792616 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:27.208653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:27.209836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:27.209836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:27.209836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:27.210464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=e7fd08895d6f4f3294c034cc88721fe1, user.id=4d5ee6e10323407785f5d83cce557bf2) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:27.210766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:27.212444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:27.218080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 531/1056] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:27 2022] GET /identity/v3/role_assignments?user.id=4d5ee6e10323407785f5d83cce557bf2&scope.project.id=e7fd08895d6f4f3294c034cc88721fe1&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:27.242778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5319ef-effa-4e35-9ca0-2128dcc699b3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:27.270417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:27.282737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:27.283400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:27.283677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:27.286686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:27.286686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] RBAC: Adding query filter params (name=image) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:27.286686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:27.286686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:27.298417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 526/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:27.299516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:27.320544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:27.407128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efe569-46ee-4d80-82a8-8a496a01a2be None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:27.440363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:27.440962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:27.441236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:27.441504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:27.442119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] RBAC: Adding query filter params (type=image) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:27.442661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:27.444051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e3b00-9fc9-4e04-90ac-2aad44a001a4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:27.453560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 532/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:27.454799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:27.558826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cba05f-d4f3-4efc-a2de-861da357068c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:27.592172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ab6a3-eb71-4a32-895d-d39f94712770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:27.592768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ab6a3-eb71-4a32-895d-d39f94712770 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:27.593041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ab6a3-eb71-4a32-895d-d39f94712770 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:27.593312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ab6a3-eb71-4a32-895d-d39f94712770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:27.593966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ab6a3-eb71-4a32-895d-d39f94712770 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:27.595348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ab6a3-eb71-4a32-895d-d39f94712770 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:27.726558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 527/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:15:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:15:28.204408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a299068-46a3-46dc-8191-4a37d9b15750 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:28.204760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a299068-46a3-46dc-8191-4a37d9b15750 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:28.204970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a299068-46a3-46dc-8191-4a37d9b15750 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:28.206354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 533/1060] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:28.211538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:28.211849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:28.212135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:28.230128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:28.231406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:28.272605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:28.304702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 528/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:28.328808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a299068-46a3-46dc-8191-4a37d9b15750 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:28.414332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:28.435904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a299068-46a3-46dc-8191-4a37d9b15750 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:28.469217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:28.469763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:28.470038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:28.470302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:28.481586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:28.486121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:28.487049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:28.488497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:28.491626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 534/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 00:15:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:28.493927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:28.591187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4240c-2eb9-47fe-91f2-3b499aa30daf None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:28.624604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:28.625199 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:28.625470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:28.625735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:28.626385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:28.626678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:28.628048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:28.636061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 529/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:15:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:28.641618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:28.744612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c717e3c-ed9c-48fd-b2ea-43194cb6d5c8 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:28.778056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:28.778675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:28.778948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:28.779213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:28.787753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:28.790644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:28.792100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:28.806434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:28.807451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:28.808369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:28.808369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:28.808369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:28.811743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 535/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:28.814123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:28.845842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:28.917018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f794b-4c7c-4317-a967-602ffd9b2285 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:28.952814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:28.953544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:28.953815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:28.954080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:28.954718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:28.955014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:28.956412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:28.968291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 530/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:28.973755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.073545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dcd6c-6a82-4af9-a938-57531f1f2ebb None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.105997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.106611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.106885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.107149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.114438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.115893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c1b12773-d928-44ce-a5ba-ef246e511ae4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:29.127036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:29.128011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:15:29.129486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:29.131573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 536/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:15:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:29.133831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.223305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:29.232940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2cf0-066e-45c7-bea9-df3a74a9cae6 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.266417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.267019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.267664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.267972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.268856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:29.269481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.270913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.279887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69145e49-0676-4f6e-9fc7-132372aaf1bc None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.280174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69145e49-0676-4f6e-9fc7-132372aaf1bc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.280453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69145e49-0676-4f6e-9fc7-132372aaf1bc None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.281933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 537/1067] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:29.288975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.288975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.288975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.289282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 531/1068] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:29.294891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.307120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:29.308333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:29.360354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:29.386803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 538/1069] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:29.400617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f51b9-c23b-4413-8649-c512defbdd8b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.408808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.440108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.441036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.441522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.441942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.454145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.455805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:29.470829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:29.471997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:29.473064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:29.473064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:29.473064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:29.474725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 532/1070] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:29.481126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.484119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:29.516783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0985a-50c2-449f-9b90-57e66372422d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.554204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.554978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.555392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.555717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.566304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:29.571185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:29.572383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:29.573249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 539/1071] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:29.578634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.593031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6c8f-96ba-48e7-b2a9-59556de6ad27 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.628032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.628561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.628838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.629107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.629753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:29.630037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.630588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:29.631465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.643280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 533/1072] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:29.649124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.700065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f1e93-75d2-451e-ac60-4732f188d204 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.720895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:29.736046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.736655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.736933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.737204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.737859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:29.738152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.739536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.749074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 540/1073] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:29.754292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.767982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a705ed4-01c8-4100-84a5-6aa64e61fae8 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.807566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.808196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.808473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.808741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.816468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.818224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-78b15752-8db3-42ba-b0e5-bca0823bdef1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:29.829745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:29.830878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:29.831854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 534/1074] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:29.832357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6122a-875f-4e78-be2d-b38822f74a69 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.832357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6122a-875f-4e78-be2d-b38822f74a69 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.832586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6122a-875f-4e78-be2d-b38822f74a69 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.833781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 535/1075] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:29.839495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.839770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.840036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.857970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:29.859157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:29.864780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb62c6-e9f3-4faf-9599-617c4cba2c6d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:29.867099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:29.898694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:29.899284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:29.900148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:29.900362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] PATH_INFO: `/v3/users/4d5ee6e10323407785f5d83cce557bf2` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:29.902326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=4d5ee6e10323407785f5d83cce557bf2)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:29.904256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:29.910533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 541/1076] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/users/4d5ee6e10323407785f5d83cce557bf2 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:29.910533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.929355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 536/1077] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:29.932120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:29.985942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:30.006434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c7a62-e728-4dbe-a840-66e337ae0ff9 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.032751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb38b4-d3c6-4101-ae5d-5a891fd12c61 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.039969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.040604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.040939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.041215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.041900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:30.042225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.043686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.054788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 542/1078] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:30.055880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:30.071764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.072398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.072680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.072951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.074067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] RBAC: Authorizing `identity:get_project(project_id=e7fd08895d6f4f3294c034cc88721fe1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.075571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.077197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 537/1079] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:29 2022] GET /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:30.077919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:30.086529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:30.155916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3676f-ff38-465a-a49f-fa6a6f796a87 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.176610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949b38b-3d42-4a33-85a4-0e3594c526bb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.188397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.188998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.189275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.189551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.195968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:30.210180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.210824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.211115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.222850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/c1031699d7da4a76965035faa9305dbd/roles/a0e5c3773fb54d60a95f39cde0a6a625` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.222850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=c1031699d7da4a76965035faa9305dbd, role_id=a0e5c3773fb54d60a95f39cde0a6a625)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.222850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.255123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.256576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.284961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:30.349842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 543/1080] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:30.350437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:30.356850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 538/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:15:30 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/c1031699d7da4a76965035faa9305dbd/roles/a0e5c3773fb54d60a95f39cde0a6a625 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:15:30.357680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:30.468950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346e04c-c081-49d4-99e6-2959a6573d68 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.491993 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57982516-c073-438d-bba4-2297b4e52b85 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.501374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:30.510716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fab6d-50fa-4e43-98a2-754b5088a84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.518183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fab6d-50fa-4e43-98a2-754b5088a84b None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.518183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fab6d-50fa-4e43-98a2-754b5088a84b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.518183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fab6d-50fa-4e43-98a2-754b5088a84b None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1/users/4d5ee6e10323407785f5d83cce557bf2/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.518183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fab6d-50fa-4e43-98a2-754b5088a84b None admin] RBAC: Authorizing `identity:create_grant(project_id=e7fd08895d6f4f3294c034cc88721fe1, user_id=4d5ee6e10323407785f5d83cce557bf2, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.518183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fab6d-50fa-4e43-98a2-754b5088a84b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.535051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.535710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.535984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.536256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.547534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.548922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:30.562798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:30.563881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:30.564857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:30.564857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:30.564857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:30.564857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 539/1082] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:30.569992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:30.598798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 544/1083] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:15:30 2022] PUT /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1/users/4d5ee6e10323407785f5d83cce557bf2/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:15:30.604450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:30.688677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a07caa-e7a1-4703-b78b-908391db87e9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:30.731658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270926ce-8536-4991-88a8-164af9563067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:30.732200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270926ce-8536-4991-88a8-164af9563067 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:30.732481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270926ce-8536-4991-88a8-164af9563067 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:30.732750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270926ce-8536-4991-88a8-164af9563067 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:30.733396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270926ce-8536-4991-88a8-164af9563067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:30.733714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270926ce-8536-4991-88a8-164af9563067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:30.735168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270926ce-8536-4991-88a8-164af9563067 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:30.747793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 540/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:32.417277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4040262-1361-45db-b026-3f027f2139a4 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.417812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4040262-1361-45db-b026-3f027f2139a4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.417812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4040262-1361-45db-b026-3f027f2139a4 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.419334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 545/1085] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:32.424296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6445-a1a8-42ca-a859-375363574df6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.424688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6445-a1a8-42ca-a859-375363574df6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.424989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6445-a1a8-42ca-a859-375363574df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.443123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bf4e6445-a1a8-42ca-a859-375363574df6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:32.444326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bf4e6445-a1a8-42ca-a859-375363574df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:32.510436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 541/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:32.533216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4040262-1361-45db-b026-3f027f2139a4 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:32.620192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9ec2-92a5-43d5-bed9-faceb0f53573 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.620515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9ec2-92a5-43d5-bed9-faceb0f53573 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.620787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed9ec2-92a5-43d5-bed9-faceb0f53573 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.622156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 542/1087] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:15:32.627135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.627452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.627725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.634581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4040262-1361-45db-b026-3f027f2139a4 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:32.645833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:32.647028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:32.663972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:32.668880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:32.669446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.669726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.669998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.680383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:32.685563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:32.686711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:32.687867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 546/1088] 10.222.0.15 () {56 vars in 1120 bytes} [Mon Aug 8 00:15:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:32.692831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:32.719115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 543/1089] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:15:32.741783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:32.745234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:32.791113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e5a30-d904-46a5-af23-e1b6682f7b88 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:32.824775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:32.825382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.825849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.826270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.827476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:32.827938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:32.829304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:32.837303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 547/1090] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:15:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:32.843043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:32.843808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab774bd-b25a-48fa-bc16-fcb683e1dd6f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:32.879118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:32.879795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.880071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.880341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.890250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:32.894830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:32.896073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:32.896960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 544/1091] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:32.902190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:32.917289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:32.946403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e330f21-9154-4bdb-b3b8-d9c49c125c08 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:32.959879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:32.980460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:32.981087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:32.981367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:32.981639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:32.991766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:32.993970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.001731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3299f45-2e97-4e7f-af3f-93cf31884845 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:33.008452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:33.009460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:33.011153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:33.011153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:33.011153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:33.011153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 548/1092] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:33.015382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.036921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.037522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.037795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.038065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.038793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:33.039091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.041102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.048985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 545/1093] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:33.054636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.114449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8de19-b85e-4cce-87da-968589713a73 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.128820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.147282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.147882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.148153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.148418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.149046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:33.149562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.150950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.152310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a8e30-c7a3-47c5-9742-dfa95ca2210f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.163191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 549/1094] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:33.169650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.187321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.187943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.188219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.188488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] PATH_INFO: `/v3/users/4d5ee6e10323407785f5d83cce557bf2` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.189611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] RBAC: Authorizing `identity:get_user(user_id=4d5ee6e10323407785f5d83cce557bf2)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.191044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.197726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 546/1095] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/users/4d5ee6e10323407785f5d83cce557bf2 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:33.197855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.198606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.266720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ec135-58d5-414c-900a-1b8d828f0ae4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.296785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802e57-428c-461c-b4e8-71f009bf9bd8 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.300787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.301407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.301684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.301958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.309043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.310478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:15:33.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:33.323360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:33.324368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 550/1096] 10.222.0.15 () {56 vars in 1093 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:33.329102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.334364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.334990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.335294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.335565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.348205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.402685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.404151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.412259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.426727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82afcba-0ed9-4780-872d-6279bc901f02 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.460154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.460677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.460955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.461226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.462487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:33.462768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.464217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.481349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 551/1097] 10.222.0.15 () {56 vars in 1133 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:33.486934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.508500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 547/1098] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:33.509534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.552684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.582674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba0231-283d-4c0a-b834-a197a1ee65cf None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.607702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d3cb6-778a-4b4b-8abf-fe4cc9bd6d44 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.618850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.619488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.619774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.620044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.634518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.641895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.642523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.642805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.643077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.644214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] RBAC: Authorizing `identity:get_project(project_id=e7fd08895d6f4f3294c034cc88721fe1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.645641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.647449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 548/1099] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:33.654329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.688599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.690036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.755134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24157fe-0646-4c7e-a4ae-8da9eb292066 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.765768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.785964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 552/1100] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 00:15:33.791098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.791098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.791098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.791098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.791448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=e7fd08895d6f4f3294c034cc88721fe1, user.id=4d5ee6e10323407785f5d83cce557bf2) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:33.791448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.792071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:33.793188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.802088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 549/1101] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/role_assignments?user.id=4d5ee6e10323407785f5d83cce557bf2&scope.project.id=e7fd08895d6f4f3294c034cc88721fe1&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:33.843055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:33.899384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608b2cb-bdc8-4d23-8fdd-7ecccc662164 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:33.934965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:33.935639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:33.935915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:33.936180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:33.947556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:33.949083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:15:33.964785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:33.965794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:15:33.966704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:15:33.966704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:15:33.966704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:33.966704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 553/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:15:33.972307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:34.072751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c71c-d1a9-4219-8582-183738674cb6 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:34.104643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:34.105243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:34.105516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:34.105780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:34.106407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:34.106704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:34.108106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:34.120711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 550/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:34.126397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:34.220133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:34.225748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd81b-bf47-4503-835b-c1dd67c8b0b0 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:34.259252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:34.260046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:34.260326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:34.260595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:34.267070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:34.268536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:15:34.279635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:34.280579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:34.281507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 554/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:34.286684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:34.320448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:34.386749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797d7bf-554a-42ac-af31-7de7b42234bc None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:34.421620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:34.422223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:34.422501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:34.422769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:34.423511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:34.423817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:34.425252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7043268-e6f3-49f7-be31-0dde64523b4d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:34.437477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 551/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:34.444047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:34.541357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4515-32c2-434b-8bde-4954f0d1d32f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:34.574194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:34.574794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:34.575074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:34.575384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:34.576124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] RBAC: Adding query filter params (role.id=a0e5c3773fb54d60a95f39cde0a6a625, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=c1031699d7da4a76965035faa9305dbd) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:34.576437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:34.577858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43683df4-94fc-48a5-b42f-387d07627306 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:34.583658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 555/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:34 2022] GET /identity/v3/role_assignments?user.id=c1031699d7da4a76965035faa9305dbd&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=a0e5c3773fb54d60a95f39cde0a6a625 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:35.847653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b41078-2ffe-44b7-aad2-f21db88c93f3 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:35.848172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b41078-2ffe-44b7-aad2-f21db88c93f3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:35.848172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b41078-2ffe-44b7-aad2-f21db88c93f3 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:35.850516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 552/1107] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:35.856172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:35.856589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:35.856748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:35.875011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:35.876218 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:35.889835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:35.940862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 556/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:35.965164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b41078-2ffe-44b7-aad2-f21db88c93f3 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:36.056524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:36.064335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b41078-2ffe-44b7-aad2-f21db88c93f3 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:36.096697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc477504-f527-4e1a-8cf0-39580faf6a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:36.097303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc477504-f527-4e1a-8cf0-39580faf6a24 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:36.097577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc477504-f527-4e1a-8cf0-39580faf6a24 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:36.097840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc477504-f527-4e1a-8cf0-39580faf6a24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:36.098918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc477504-f527-4e1a-8cf0-39580faf6a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:36.100351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc477504-f527-4e1a-8cf0-39580faf6a24 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:36.101884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 553/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:36.108018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:36.202691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1013-328c-46f3-8b0b-706413959832 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:36.235123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5848fe-13d3-44f3-829c-a0d5ef833ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:36.235738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5848fe-13d3-44f3-829c-a0d5ef833ba0 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:36.236012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5848fe-13d3-44f3-829c-a0d5ef833ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:36.236280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5848fe-13d3-44f3-829c-a0d5ef833ba0 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:36.236912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5848fe-13d3-44f3-829c-a0d5ef833ba0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:36.238316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5848fe-13d3-44f3-829c-a0d5ef833ba0 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:36.334849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 557/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:15:36 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:15:38.369133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5d009-8a55-422a-9611-fab164779dbc None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:38.369730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5d009-8a55-422a-9611-fab164779dbc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:38.369730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5d009-8a55-422a-9611-fab164779dbc None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:38.370995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 554/1111] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:38.376368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:38.376804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:38.376905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:38.390237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:38.395540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:38.396693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:38.461044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 558/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:38.484942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5d009-8a55-422a-9611-fab164779dbc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:38.588920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5d009-8a55-422a-9611-fab164779dbc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:38.628683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:38.629245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:38.629524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:38.629796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] PATH_INFO: `/v3/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:38.631174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] RBAC: Authorizing `identity:get_role(role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:38.632553 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:38.633889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 555/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:38 2022] GET /identity/v3/roles/1cef455d27914904ba2d1176ced4233c => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:38.639856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:38.736598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7484b-7fbb-4bad-8a6a-5c22a7edbb9d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:38.768458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:38.769080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:38.769358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:38.769633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] PATH_INFO: `/v3/users/0340bf027c0142159f6e5d0bb2dd8b24` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:38.772238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:38.782914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] RBAC: Authorizing `identity:get_user(user_id=0340bf027c0142159f6e5d0bb2dd8b24)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:38.786210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:38.786210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 559/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:38 2022] GET /identity/v3/users/0340bf027c0142159f6e5d0bb2dd8b24 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:38.792526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:38.822130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:38.888341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53a531-4b32-4627-96d8-893279f3fd69 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:38.920800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:38.921388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:38.921662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:38.921929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:38.987560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:38.988988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:39.016525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:39.083566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 556/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 00:15:39.090324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:39.185144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5aa11-3b25-40d8-a381-a369b28acac7 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:39.218713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57514c6-e5fd-492d-93c8-77b20697d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:39.219382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57514c6-e5fd-492d-93c8-77b20697d947 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:39.219694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57514c6-e5fd-492d-93c8-77b20697d947 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:39.219962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57514c6-e5fd-492d-93c8-77b20697d947 None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:39.221058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57514c6-e5fd-492d-93c8-77b20697d947 None admin] RBAC: Authorizing `identity:get_project(project_id=e7fd08895d6f4f3294c034cc88721fe1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:39.222459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57514c6-e5fd-492d-93c8-77b20697d947 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:39.224072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 560/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:39 2022] GET /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:39.230523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:39.328527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049cdd2-1137-4e2d-bbeb-1b81a51c8062 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:39.355214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:39.360938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:39.361533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:39.361810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:39.362079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:39.362794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] RBAC: Adding query filter params (role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=e7fd08895d6f4f3294c034cc88721fe1, user.id=0340bf027c0142159f6e5d0bb2dd8b24) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:39.363098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:39.364541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccc423-ea05-4209-b56c-f6653ab7e0e3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:39.370298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 557/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:39 2022] GET /identity/v3/role_assignments?user.id=0340bf027c0142159f6e5d0bb2dd8b24&scope.project.id=e7fd08895d6f4f3294c034cc88721fe1&role.id=1cef455d27914904ba2d1176ced4233c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:41.311666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963aa3e-05ba-4c18-8937-90d181db46dd None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:41.312242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963aa3e-05ba-4c18-8937-90d181db46dd None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:41.312242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963aa3e-05ba-4c18-8937-90d181db46dd None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:41.313524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 561/1118] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:41.318938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:41.319399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:41.319585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:41.338643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:41.339853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:41.404168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 558/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:41.425984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963aa3e-05ba-4c18-8937-90d181db46dd None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:41.486403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:41.524710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963aa3e-05ba-4c18-8937-90d181db46dd None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:41.556651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:41.557254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:41.557531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:41.557801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] PATH_INFO: `/v3/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:41.559156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] RBAC: Authorizing `identity:get_role(role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:41.560530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:41.561853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 562/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:41 2022] GET /identity/v3/roles/1cef455d27914904ba2d1176ced4233c => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:41.567678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:41.665543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590c04f-6b38-4126-be90-47e16767b259 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:41.688936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:41.700232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:41.705940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:41.705940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:41.705940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] PATH_INFO: `/v3/users/0340bf027c0142159f6e5d0bb2dd8b24` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:41.705940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=0340bf027c0142159f6e5d0bb2dd8b24)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:41.705940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:41.705940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 559/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:41 2022] GET /identity/v3/users/0340bf027c0142159f6e5d0bb2dd8b24 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:41.711338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:41.807177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab4553-4e63-4586-9599-c583e8705261 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:41.834895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:41.840568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929da7b-6f24-4a61-9e5d-eb5bdbf55191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:41.846867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929da7b-6f24-4a61-9e5d-eb5bdbf55191 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:41.846867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929da7b-6f24-4a61-9e5d-eb5bdbf55191 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:41.846867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929da7b-6f24-4a61-9e5d-eb5bdbf55191 None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:41.846867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929da7b-6f24-4a61-9e5d-eb5bdbf55191 None admin] RBAC: Authorizing `identity:get_project(project_id=e7fd08895d6f4f3294c034cc88721fe1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:41.846867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929da7b-6f24-4a61-9e5d-eb5bdbf55191 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:41.846867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 563/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:41 2022] GET /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:41.851767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:41.950003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc87dade-e31a-4280-b52a-c3c8518bc1d2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:41.982040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161e041-032e-42f2-8f1c-5b30718564e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:41.982682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161e041-032e-42f2-8f1c-5b30718564e9 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:41.982956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161e041-032e-42f2-8f1c-5b30718564e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:41.983234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161e041-032e-42f2-8f1c-5b30718564e9 None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1/users/0340bf027c0142159f6e5d0bb2dd8b24/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:41.985348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161e041-032e-42f2-8f1c-5b30718564e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e7fd08895d6f4f3294c034cc88721fe1, user_id=0340bf027c0142159f6e5d0bb2dd8b24, role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:41.986764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161e041-032e-42f2-8f1c-5b30718564e9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:42.111249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 560/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:15:41 2022] PUT /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1/users/0340bf027c0142159f6e5d0bb2dd8b24/roles/1cef455d27914904ba2d1176ced4233c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:15:44.080057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc9d9-dd5e-4f5d-a81c-1ea26dc7a70c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:44.080559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc9d9-dd5e-4f5d-a81c-1ea26dc7a70c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:44.080559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc9d9-dd5e-4f5d-a81c-1ea26dc7a70c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:44.081851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 564/1124] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:44.087005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:44.087404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:44.087600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:44.105634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:44.106807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:44.152258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:44.171446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 561/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:44.195213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc9d9-dd5e-4f5d-a81c-1ea26dc7a70c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:44.291796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc9d9-dd5e-4f5d-a81c-1ea26dc7a70c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:44.324732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:44.325338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:44.325610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:44.325875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] PATH_INFO: `/v3/roles/1cef455d27914904ba2d1176ced4233c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:44.327253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] RBAC: Authorizing `identity:get_role(role_id=1cef455d27914904ba2d1176ced4233c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:44.328589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:44.329872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 565/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:44 2022] GET /identity/v3/roles/1cef455d27914904ba2d1176ced4233c => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:44.335734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:44.430716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e9d0f-b8b9-4dc8-8b4c-2d02ec7fc980 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:44.464503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:44.465121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:44.465397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:44.465667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] PATH_INFO: `/v3/users/0340bf027c0142159f6e5d0bb2dd8b24` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:44.466771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] RBAC: Authorizing `identity:get_user(user_id=0340bf027c0142159f6e5d0bb2dd8b24)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:44.468223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:44.469788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 562/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:44 2022] GET /identity/v3/users/0340bf027c0142159f6e5d0bb2dd8b24 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:15:44.475631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:44.538016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:44.571599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b82424-9c1e-4944-9a61-264a0e4742a5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:44.607116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:44.607739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:44.608018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:44.608290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:44.672383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:44.673834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:44.734480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:44.767305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 566/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:44.773229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:44.822927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:44.871208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea749426-2b7b-4757-891e-71867ed0c591 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:44.903674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7647f99-1e7d-4d8d-8b08-de27675a285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:44.904282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7647f99-1e7d-4d8d-8b08-de27675a285f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:44.904559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7647f99-1e7d-4d8d-8b08-de27675a285f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:44.904832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7647f99-1e7d-4d8d-8b08-de27675a285f None admin] PATH_INFO: `/v3/projects/e7fd08895d6f4f3294c034cc88721fe1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:44.905921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7647f99-1e7d-4d8d-8b08-de27675a285f None admin] RBAC: Authorizing `identity:get_project(project_id=e7fd08895d6f4f3294c034cc88721fe1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:44.907384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7647f99-1e7d-4d8d-8b08-de27675a285f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:44.908994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 563/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:44 2022] GET /identity/v3/projects/e7fd08895d6f4f3294c034cc88721fe1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:44.915126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:45.009669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92908f-fcb8-4cdc-a2cf-dc26b1cb5eb4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:45.042850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:45.043462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:45.043740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:45.044009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:45.044715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] RBAC: Adding query filter params (role.id=1cef455d27914904ba2d1176ced4233c, scope.project.id=e7fd08895d6f4f3294c034cc88721fe1, user.id=0340bf027c0142159f6e5d0bb2dd8b24) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:45.045020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:45.046431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7cd965-8eb8-4756-8d77-c835f20436cc None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:45.052007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 567/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:44 2022] GET /identity/v3/role_assignments?user.id=0340bf027c0142159f6e5d0bb2dd8b24&scope.project.id=e7fd08895d6f4f3294c034cc88721fe1&role.id=1cef455d27914904ba2d1176ced4233c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:47.048805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d798-6a20-4738-bcb7-92962325889f None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:47.049317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d798-6a20-4738-bcb7-92962325889f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:47.049317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a10d798-6a20-4738-bcb7-92962325889f None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:47.050641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 564/1131] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:47.055837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:47.056254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:47.056382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:47.075021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:47.076236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:47.099378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:47.141287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 568/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:47.164651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d798-6a20-4738-bcb7-92962325889f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:47.264402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a10d798-6a20-4738-bcb7-92962325889f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:47.281784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:47.298086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9645c6-73ea-45a1-9c35-49ce28ae835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:47.298703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9645c6-73ea-45a1-9c35-49ce28ae835a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:47.299029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9645c6-73ea-45a1-9c35-49ce28ae835a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:47.299272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9645c6-73ea-45a1-9c35-49ce28ae835a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:47.300376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9645c6-73ea-45a1-9c35-49ce28ae835a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:47.301818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9645c6-73ea-45a1-9c35-49ce28ae835a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:47.303405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 565/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:47.309550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:47.407213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db042a-d699-4c04-9265-1c5defff4274 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:47.440228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52c8aa-268e-4d21-8f9d-10ebbd0d4890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:47.440820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c8aa-268e-4d21-8f9d-10ebbd0d4890 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:47.441093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c8aa-268e-4d21-8f9d-10ebbd0d4890 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:47.441360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c8aa-268e-4d21-8f9d-10ebbd0d4890 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:47.441983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c8aa-268e-4d21-8f9d-10ebbd0d4890 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:47.443684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c8aa-268e-4d21-8f9d-10ebbd0d4890 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:47.526470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 569/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:15:47 2022] POST /identity/v3/projects => generated 311 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:49.536841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec565e6c-b3b5-47c1-8de7-c876339be812 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:49.537370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec565e6c-b3b5-47c1-8de7-c876339be812 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:49.537370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec565e6c-b3b5-47c1-8de7-c876339be812 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:49.538734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 566/1135] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:49.544461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:49.544869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:49.545000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:49.563801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:49.564973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:49.572168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:49.629045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 570/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:49.653194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec565e6c-b3b5-47c1-8de7-c876339be812 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:49.752565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec565e6c-b3b5-47c1-8de7-c876339be812 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:49.784912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f99109-0442-45d6-b6e8-f8cb2023c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:49.785526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f99109-0442-45d6-b6e8-f8cb2023c768 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:49.785800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f99109-0442-45d6-b6e8-f8cb2023c768 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:49.786067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f99109-0442-45d6-b6e8-f8cb2023c768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:49.787201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f99109-0442-45d6-b6e8-f8cb2023c768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:49.788601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f99109-0442-45d6-b6e8-f8cb2023c768 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:49.790152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 567/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:15:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:49.796660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:49.890199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3907e226-3d4e-49ae-9dc0-0b17da5c92e8 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:49.923197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf62b1c-d07d-4a45-bb1b-8f4e3248ff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:49.923872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf62b1c-d07d-4a45-bb1b-8f4e3248ff84 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:49.924148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf62b1c-d07d-4a45-bb1b-8f4e3248ff84 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:49.924419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf62b1c-d07d-4a45-bb1b-8f4e3248ff84 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:49.925054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf62b1c-d07d-4a45-bb1b-8f4e3248ff84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:49.926459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf62b1c-d07d-4a45-bb1b-8f4e3248ff84 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:50.020787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:50.026799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 571/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:15:49 2022] POST /identity/v3/users => generated 286 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:15:52.065071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ee35-fafb-4617-a8e8-b3b641cb7396 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:52.065589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ee35-fafb-4617-a8e8-b3b641cb7396 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:52.065589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29ee35-fafb-4617-a8e8-b3b641cb7396 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:52.066864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 568/1139] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:52.072767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:52.073301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:52.073301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:52.090922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:52.092130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:52.155797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 572/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:52.178352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ee35-fafb-4617-a8e8-b3b641cb7396 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:52.233361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:52.277941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29ee35-fafb-4617-a8e8-b3b641cb7396 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:52.311279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:52.311889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:52.312164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:52.312433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:52.324090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:52.328684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:52.329867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:52.330982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 569/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:52.336002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:52.430930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f12234-7d24-4af9-bdf6-66991dba5fc9 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:52.456620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:52.464165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:52.464758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:52.465033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:52.470138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:52.470138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:52.470138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:52.470138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:52.475331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 573/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:52.481666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:52.580017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f7968-e61d-4b43-9f7f-d536e67d9755 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:52.593030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:52.614916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:52.615562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:52.615866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:52.616161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] PATH_INFO: `/v3/users/ecc24fbea0d04150a61d514ed71ea2ae` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:52.629971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] RBAC: Authorizing `identity:get_user(user_id=ecc24fbea0d04150a61d514ed71ea2ae)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:52.631461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:52.633074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 570/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:52 2022] GET /identity/v3/users/ecc24fbea0d04150a61d514ed71ea2ae => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:52.639009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:52.737063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bb566-9c94-4eee-b75f-19421d846a37 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:52.768977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:52.769568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:52.769853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:52.770123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:52.825676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:52.838771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:52.838771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:52.932312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 574/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:52.938344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:53.103694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52b11-11e1-4073-8d61-96a4f2466729 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:53.136065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85058b1-de22-4ac2-b6e4-92938df77cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:53.136672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85058b1-de22-4ac2-b6e4-92938df77cd9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:53.136946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85058b1-de22-4ac2-b6e4-92938df77cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:53.137215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85058b1-de22-4ac2-b6e4-92938df77cd9 None admin] PATH_INFO: `/v3/projects/ea603a4ae6f944edabeb481022700cd0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:53.138300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85058b1-de22-4ac2-b6e4-92938df77cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=ea603a4ae6f944edabeb481022700cd0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:53.139746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85058b1-de22-4ac2-b6e4-92938df77cd9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:53.141346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 571/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:52 2022] GET /identity/v3/projects/ea603a4ae6f944edabeb481022700cd0 => generated 311 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:53.147476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:53.232318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:53.241294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc3fcd-a709-4826-9491-165630fbd412 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:53.273759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:53.274347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:53.274621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:53.274886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:53.276196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=ea603a4ae6f944edabeb481022700cd0, user.id=ecc24fbea0d04150a61d514ed71ea2ae) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:53.276494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:53.277892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc86541-2737-405e-812f-0890a11a9aa2 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:53.282998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 575/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:53 2022] GET /identity/v3/role_assignments?user.id=ecc24fbea0d04150a61d514ed71ea2ae&scope.project.id=ea603a4ae6f944edabeb481022700cd0&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:55.250372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b46f0-24bc-4160-99ef-2668efbe09a7 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:55.250963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b46f0-24bc-4160-99ef-2668efbe09a7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:55.250963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b46f0-24bc-4160-99ef-2668efbe09a7 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:55.252280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 572/1147] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:55.257343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:55.257727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:55.257884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:55.276031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:55.277287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:55.347281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 576/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:55.370751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b46f0-24bc-4160-99ef-2668efbe09a7 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:55.412117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:55.469463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b46f0-24bc-4160-99ef-2668efbe09a7 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:55.503041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:55.503621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:55.503899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:55.504169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:55.514001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:55.518583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:55.519854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:15:55.521103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 573/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:15:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:55.525828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:55.606816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:55.622383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a44573-6ab2-405c-a853-f4bddd306080 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:55.655460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:55.656051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:55.656325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:55.656592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:55.657242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:55.657540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:55.658885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:55.667476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 577/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:55.673224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:55.767503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:55.770290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b06b3-0584-4554-994d-14b990f54948 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:55.803546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:55.804156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:55.804433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:55.804702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] PATH_INFO: `/v3/users/ecc24fbea0d04150a61d514ed71ea2ae` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:55.805800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] RBAC: Authorizing `identity:get_user(user_id=ecc24fbea0d04150a61d514ed71ea2ae)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:55.807391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:55.808930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 574/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:55 2022] GET /identity/v3/users/ecc24fbea0d04150a61d514ed71ea2ae => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:55.814571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:55.912739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756884b5-63b9-4b24-af24-6968ab09ce1b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:55.945597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a33a9-140e-419d-9312-191509390044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:55.946211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a33a9-140e-419d-9312-191509390044 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:55.946485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a33a9-140e-419d-9312-191509390044 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:55.946753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a33a9-140e-419d-9312-191509390044 None admin] PATH_INFO: `/v3/projects/ea603a4ae6f944edabeb481022700cd0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:55.948053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a33a9-140e-419d-9312-191509390044 None admin] RBAC: Authorizing `identity:get_project(project_id=ea603a4ae6f944edabeb481022700cd0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:55.949524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a33a9-140e-419d-9312-191509390044 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:55.951125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 578/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:55 2022] GET /identity/v3/projects/ea603a4ae6f944edabeb481022700cd0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:55.957095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:56.058430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb71081-6a2d-478b-990e-225f10d24ded None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:56.092073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d31a93-d2a7-491b-8784-edc00dd3f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:56.092717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d31a93-d2a7-491b-8784-edc00dd3f934 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:56.092998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d31a93-d2a7-491b-8784-edc00dd3f934 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:56.093272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d31a93-d2a7-491b-8784-edc00dd3f934 None admin] PATH_INFO: `/v3/projects/ea603a4ae6f944edabeb481022700cd0/users/ecc24fbea0d04150a61d514ed71ea2ae/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:56.095256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d31a93-d2a7-491b-8784-edc00dd3f934 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea603a4ae6f944edabeb481022700cd0, user_id=ecc24fbea0d04150a61d514ed71ea2ae, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:56.096765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d31a93-d2a7-491b-8784-edc00dd3f934 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:56.188775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 575/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:15:55 2022] PUT /identity/v3/projects/ea603a4ae6f944edabeb481022700cd0/users/ecc24fbea0d04150a61d514ed71ea2ae/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:15:58.149745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a8497-14a6-47a4-87aa-57eaed7558f8 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:58.150740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a8497-14a6-47a4-87aa-57eaed7558f8 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:58.151040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a8497-14a6-47a4-87aa-57eaed7558f8 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:58.152515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 579/1154] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:15:58.157548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:58.157950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:58.158101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:58.176978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:15:58.178143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:15:58.200482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:58.242781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 576/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:15:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:15:58.266194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a8497-14a6-47a4-87aa-57eaed7558f8 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:58.282380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:58.366171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a8497-14a6-47a4-87aa-57eaed7558f8 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:58.398068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:58.398680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:58.398959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:58.399258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:58.409997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:15:58.414543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:15:58.415628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:15:58.416503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 580/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:15:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:15:58.421813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:58.516658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301f1c1-3e74-4cf7-bb65-759d71f8e81f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:58.548900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:58.549489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:58.549767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:58.550039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:58.550686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:58.550984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:58.552547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:58.560408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 577/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:15:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:58.566165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:58.638985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:58.663290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018499ac-7673-43cb-b8f5-75aec5c8c977 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:58.696705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08439f-9092-4149-882b-936320390515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:58.697868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08439f-9092-4149-882b-936320390515 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:58.698142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08439f-9092-4149-882b-936320390515 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:58.698415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08439f-9092-4149-882b-936320390515 None admin] PATH_INFO: `/v3/users/ecc24fbea0d04150a61d514ed71ea2ae` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:58.699559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08439f-9092-4149-882b-936320390515 None admin] RBAC: Authorizing `identity:get_user(user_id=ecc24fbea0d04150a61d514ed71ea2ae)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:58.700981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08439f-9092-4149-882b-936320390515 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:58.702580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 581/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:15:58 2022] GET /identity/v3/users/ecc24fbea0d04150a61d514ed71ea2ae => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:58.708356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:58.730244 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:58.805712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f2270-17fb-44a7-bf43-9e9207264c68 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:58.839734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:58.840320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:58.840597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:58.840867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:58.904846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:58.906287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:58.925542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:59.001310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 578/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:15:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:15:59.007287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08439f-9092-4149-882b-936320390515 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:59.115366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08439f-9092-4149-882b-936320390515 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:59.147192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd18f7b-3fd8-43e3-9645-00711652aa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:59.147831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd18f7b-3fd8-43e3-9645-00711652aa48 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:59.148104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd18f7b-3fd8-43e3-9645-00711652aa48 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:59.148374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd18f7b-3fd8-43e3-9645-00711652aa48 None admin] PATH_INFO: `/v3/projects/ea603a4ae6f944edabeb481022700cd0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:59.149463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd18f7b-3fd8-43e3-9645-00711652aa48 None admin] RBAC: Authorizing `identity:get_project(project_id=ea603a4ae6f944edabeb481022700cd0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:59.150901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd18f7b-3fd8-43e3-9645-00711652aa48 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:59.152520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 582/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:15:59 2022] GET /identity/v3/projects/ea603a4ae6f944edabeb481022700cd0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:15:59.158607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:15:59.255006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df3883-c0af-46b3-b486-4dbb27896e58 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:15:59.273670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:15:59.287105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:15:59.287720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:15:59.287995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:15:59.288262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:15:59.288966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=ea603a4ae6f944edabeb481022700cd0, user.id=ecc24fbea0d04150a61d514ed71ea2ae) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:15:59.289265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:15:59.291284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cba75c-8284-43fd-b174-d25e4907ae80 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:15:59.296971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 579/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:15:59 2022] GET /identity/v3/role_assignments?user.id=ecc24fbea0d04150a61d514ed71ea2ae&scope.project.id=ea603a4ae6f944edabeb481022700cd0&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:01.284255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f24b0-6ab2-4843-b0de-0e011e73ba21 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:01.284805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f24b0-6ab2-4843-b0de-0e011e73ba21 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:01.284805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f24b0-6ab2-4843-b0de-0e011e73ba21 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:01.286126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 583/1162] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:01.291399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:01.291818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:01.291942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:01.310817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:01.312055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:01.375386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 580/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:16:01.397354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f24b0-6ab2-4843-b0de-0e011e73ba21 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:01.410171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:01.494871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f24b0-6ab2-4843-b0de-0e011e73ba21 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:01.526972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:01.527579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:01.527854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:01.528119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:01.591738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:01.593168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:01.603396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:01.686449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 584/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:16:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:16:01.692289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:01.787517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df00749-471e-4e59-a118-f6ac4db5bec8 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:01.800786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:01.820245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:01.820855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:01.821129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:01.821398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:01.828611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:01.830020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-72368869-9910-4715-9177-0ecb83b0bf81 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:16:01.841180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:16:01.842404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:16:01.843328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 581/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:16:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:16:01.848184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:01.942907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1afec-afd1-4480-a00d-dbb1dea8d00a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:01.954042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:01.977115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:01.977736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:01.978063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:01.978400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:01.979027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:16:01.979345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:01.980690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c39a2a-214d-412e-8d91-6844619ce357 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:01.988182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 585/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Mon Aug 8 00:16:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:03.964741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514d447-2041-43f0-a2dc-a11f90415ffe None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:03.965238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514d447-2041-43f0-a2dc-a11f90415ffe None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:03.965238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514d447-2041-43f0-a2dc-a11f90415ffe None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:03.966533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 582/1167] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:03.971776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67362-6088-490b-9d4a-f67d02445540 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:03.972196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67362-6088-490b-9d4a-f67d02445540 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:03.972324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67362-6088-490b-9d4a-f67d02445540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:03.989821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-57e67362-6088-490b-9d4a-f67d02445540 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:03.991004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-57e67362-6088-490b-9d4a-f67d02445540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:04.055860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 586/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:16:04.079529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514d447-2041-43f0-a2dc-a11f90415ffe None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:04.176439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514d447-2041-43f0-a2dc-a11f90415ffe None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:04.209829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8fa08-e0d9-483d-8fe8-1f9ad320c70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:04.210417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8fa08-e0d9-483d-8fe8-1f9ad320c70b None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:04.210693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8fa08-e0d9-483d-8fe8-1f9ad320c70b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:04.210962 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8fa08-e0d9-483d-8fe8-1f9ad320c70b None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:04.211647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8fa08-e0d9-483d-8fe8-1f9ad320c70b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:04.213041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8fa08-e0d9-483d-8fe8-1f9ad320c70b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:04.292763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 583/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Mon Aug 8 00:16:04 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:16:06.318901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28f612-1214-4bc9-bfe3-ea26a858f58a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:06.319456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28f612-1214-4bc9-bfe3-ea26a858f58a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:06.319456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28f612-1214-4bc9-bfe3-ea26a858f58a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:06.320812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 587/1170] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:06.326590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:06.327012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:06.327167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:06.344992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:06.346153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:06.372297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:06.412636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 584/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:16:06.436475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28f612-1214-4bc9-bfe3-ea26a858f58a None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:06.533226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28f612-1214-4bc9-bfe3-ea26a858f58a None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:06.535485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:06.565727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11888cd3-9ba7-4faa-a259-5466989ce80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:06.566364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888cd3-9ba7-4faa-a259-5466989ce80a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:06.566640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888cd3-9ba7-4faa-a259-5466989ce80a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:06.566904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11888cd3-9ba7-4faa-a259-5466989ce80a None admin] PATH_INFO: `/v3/domains/8ab00a59c64440db9a049f721ad14acc` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:06.578165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11888cd3-9ba7-4faa-a259-5466989ce80a None admin] RBAC: Authorizing `identity:get_domain(domain_id=8ab00a59c64440db9a049f721ad14acc)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:06.579564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11888cd3-9ba7-4faa-a259-5466989ce80a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:06.581120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 588/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Aug 8 00:16:06 2022] GET /identity/v3/domains/8ab00a59c64440db9a049f721ad14acc => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:06.587021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:06.685489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08cf23-7065-4179-a63c-3710b710c5ce None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:06.718705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8fdde-a6bb-49d0-9f30-3c5a9b3667e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:06.719353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8fdde-a6bb-49d0-9f30-3c5a9b3667e5 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:06.719638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8fdde-a6bb-49d0-9f30-3c5a9b3667e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:06.719915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8fdde-a6bb-49d0-9f30-3c5a9b3667e5 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:06.720570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8fdde-a6bb-49d0-9f30-3c5a9b3667e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:06.722005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8fdde-a6bb-49d0-9f30-3c5a9b3667e5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:06.838979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 585/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:16:06 2022] POST /identity/v3/projects => generated 361 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:16:08.881299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee60d9-8aad-4e91-bf90-acc7011e480c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:08.881882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee60d9-8aad-4e91-bf90-acc7011e480c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:08.881882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee60d9-8aad-4e91-bf90-acc7011e480c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:08.883189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 589/1174] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:08.888441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:08.888874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:08.888982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:08.907212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:08.908412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:08.919086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:08.972421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 586/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:16:08.997561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee60d9-8aad-4e91-bf90-acc7011e480c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:09.094413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee60d9-8aad-4e91-bf90-acc7011e480c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:09.126727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcea2cd-995f-45f6-b3f0-f5c46e414ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:09.127377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcea2cd-995f-45f6-b3f0-f5c46e414ede None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:09.127656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcea2cd-995f-45f6-b3f0-f5c46e414ede None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:09.127924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcea2cd-995f-45f6-b3f0-f5c46e414ede None admin] PATH_INFO: `/v3/domains/8ab00a59c64440db9a049f721ad14acc` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:09.129049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcea2cd-995f-45f6-b3f0-f5c46e414ede None admin] RBAC: Authorizing `identity:get_domain(domain_id=8ab00a59c64440db9a049f721ad14acc)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:09.130463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcea2cd-995f-45f6-b3f0-f5c46e414ede None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:09.132030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 590/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Mon Aug 8 00:16:08 2022] GET /identity/v3/domains/8ab00a59c64440db9a049f721ad14acc => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:09.137668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:09.233087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdf1f2-85d0-42aa-b1bd-30ad432d8c40 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:09.264942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46440dd0-2468-474a-9175-a163fbeee822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:09.265531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440dd0-2468-474a-9175-a163fbeee822 None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:09.265805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440dd0-2468-474a-9175-a163fbeee822 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:09.266074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46440dd0-2468-474a-9175-a163fbeee822 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:09.266698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46440dd0-2468-474a-9175-a163fbeee822 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:09.268169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46440dd0-2468-474a-9175-a163fbeee822 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:09.374994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 587/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:16:09 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:16:11.396815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ee5ab-564c-4d68-a060-c8d4ca2c911b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:11.397387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ee5ab-564c-4d68-a060-c8d4ca2c911b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:11.397387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ee5ab-564c-4d68-a060-c8d4ca2c911b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:11.398705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 591/1178] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:11.403991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:11.404365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:11.404547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:11.410734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:11.423052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:11.424260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:11.489261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 588/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:16:11.512074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ee5ab-564c-4d68-a060-c8d4ca2c911b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:11.543665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:11.612753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ee5ab-564c-4d68-a060-c8d4ca2c911b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:11.644138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:11.644755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:11.645032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:11.645304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:11.656213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:16:11.660782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:16:11.661949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:16:11.663126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 592/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:16:11.668701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:11.763749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a309b-4f35-41d7-84e9-5035fc38148f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:11.792456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:11.797084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:11.797674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:11.797949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:11.798216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:11.798867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:16:11.799166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:11.800547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:11.808484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 589/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:11.814089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:11.902078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:11.911409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bd5c3-b815-4035-9550-9566be5cccd6 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:11.942050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:11.942662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:11.942938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:11.943207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] PATH_INFO: `/v3/users/01cf91e2257a41718ef9051b804d7bb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:11.956833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] RBAC: Authorizing `identity:get_user(user_id=01cf91e2257a41718ef9051b804d7bb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:11.958264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:11.959875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 593/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:16:11 2022] GET /identity/v3/users/01cf91e2257a41718ef9051b804d7bb0 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:11.966287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:12.062459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b164b-4dd5-44cd-903a-5179d24c88f3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:12.095620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:12.096208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:12.096486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:12.096756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:12.155132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:12.168805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:12.168805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:12.259263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 590/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:16:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:16:12.265629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:12.360967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f277e-a500-45e8-83ac-8f0b5142e805 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:12.394903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b879640-1f94-47d6-894f-561f28ac62fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:12.395541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b879640-1f94-47d6-894f-561f28ac62fb None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:12.395819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b879640-1f94-47d6-894f-561f28ac62fb None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:12.396088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b879640-1f94-47d6-894f-561f28ac62fb None admin] PATH_INFO: `/v3/projects/2a9460fd27ca4e55b47db9592bab6c22` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:12.397175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b879640-1f94-47d6-894f-561f28ac62fb None admin] RBAC: Authorizing `identity:get_project(project_id=2a9460fd27ca4e55b47db9592bab6c22)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:12.398592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b879640-1f94-47d6-894f-561f28ac62fb None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:12.400242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 594/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:16:12 2022] GET /identity/v3/projects/2a9460fd27ca4e55b47db9592bab6c22 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:12.406280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:12.496771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:12.503138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb75167-94f6-41dd-9d6e-e70c3b2595b3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:12.536380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:12.536965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:12.537238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:12.537504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:12.538211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=2a9460fd27ca4e55b47db9592bab6c22, user.id=01cf91e2257a41718ef9051b804d7bb0) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:16:12.538509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:12.539939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53400c-e7b3-4465-a70b-54dfb36a6c46 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:12.545242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 591/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:16:12 2022] GET /identity/v3/role_assignments?user.id=01cf91e2257a41718ef9051b804d7bb0&scope.project.id=2a9460fd27ca4e55b47db9592bab6c22&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:14.498055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be587f-5b02-422c-897a-8c390dc891cb None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:14.498612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be587f-5b02-422c-897a-8c390dc891cb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:14.498612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be587f-5b02-422c-897a-8c390dc891cb None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:14.499956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 595/1186] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:14.505211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:14.505582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:14.505767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:14.523586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:14.525274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:14.590072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 592/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:16:14.612734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be587f-5b02-422c-897a-8c390dc891cb None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:14.631137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:14.713650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be587f-5b02-422c-897a-8c390dc891cb None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:14.746763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:14.747403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:14.747684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:14.747988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:14.759808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:16:14.764408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:16:14.765516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:16:14.766630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 596/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:16:14.771629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:14.856154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:14.867408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d6d58c-3312-40ba-bb74-c18e8de7839a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:14.900435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:14.901152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:14.901428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:14.901700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:14.902353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:16:14.902652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:14.904034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:14.911748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 593/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:14.917677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:14.992211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:15.012788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea872e-93cf-43b8-9401-3df7e5061365 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:15.045974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:15.046584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:15.046857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:15.047125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] PATH_INFO: `/v3/users/01cf91e2257a41718ef9051b804d7bb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:15.048240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] RBAC: Authorizing `identity:get_user(user_id=01cf91e2257a41718ef9051b804d7bb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:15.049649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:15.051171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 597/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:16:14 2022] GET /identity/v3/users/01cf91e2257a41718ef9051b804d7bb0 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:15.057117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:15.155700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77ae43-d570-4168-8ff2-e9681cf5f1ac None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:15.187796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4fbe8-a3c9-48fe-bd89-b1bc4280d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:15.188407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4fbe8-a3c9-48fe-bd89-b1bc4280d3f7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:15.188687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4fbe8-a3c9-48fe-bd89-b1bc4280d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:15.188960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4fbe8-a3c9-48fe-bd89-b1bc4280d3f7 None admin] PATH_INFO: `/v3/projects/2a9460fd27ca4e55b47db9592bab6c22` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:15.190062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4fbe8-a3c9-48fe-bd89-b1bc4280d3f7 None admin] RBAC: Authorizing `identity:get_project(project_id=2a9460fd27ca4e55b47db9592bab6c22)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:15.191514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4fbe8-a3c9-48fe-bd89-b1bc4280d3f7 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:15.193132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 594/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:16:15 2022] GET /identity/v3/projects/2a9460fd27ca4e55b47db9592bab6c22 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:15.199796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:15.295748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acde83-3c4d-4498-bb1a-eb3d0808b67b None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:15.329041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d66c7e-8b31-4187-98a6-e756c5f34417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:15.329681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d66c7e-8b31-4187-98a6-e756c5f34417 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:15.329957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d66c7e-8b31-4187-98a6-e756c5f34417 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:15.330230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d66c7e-8b31-4187-98a6-e756c5f34417 None admin] PATH_INFO: `/v3/projects/2a9460fd27ca4e55b47db9592bab6c22/users/01cf91e2257a41718ef9051b804d7bb0/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:15.332138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d66c7e-8b31-4187-98a6-e756c5f34417 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a9460fd27ca4e55b47db9592bab6c22, user_id=01cf91e2257a41718ef9051b804d7bb0, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:15.333562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d66c7e-8b31-4187-98a6-e756c5f34417 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:15.405998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 598/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:16:15 2022] PUT /identity/v3/projects/2a9460fd27ca4e55b47db9592bab6c22/users/01cf91e2257a41718ef9051b804d7bb0/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:16:17.363788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c019759-b598-4fa8-9f59-3751bbf8bd6e None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:17.364360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c019759-b598-4fa8-9f59-3751bbf8bd6e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:17.364360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c019759-b598-4fa8-9f59-3751bbf8bd6e None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:17.365666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 595/1193] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:16:17.370857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:17.371281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:17.371425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:17.383434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:17.389935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:16:17.391106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:16:17.456890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 599/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:16:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:16:17.481169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c019759-b598-4fa8-9f59-3751bbf8bd6e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:17.500467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:17.580480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c019759-b598-4fa8-9f59-3751bbf8bd6e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:17.612535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:17.613146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:17.613503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:17.613777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:17.623945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:16:17.628556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:16:17.629660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:16:17.630703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 596/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:16:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:16:17.636296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:17.729631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dbcbc-5d37-4a28-87f1-6ddce3f7c851 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:17.762213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:17.762800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:17.763075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:17.763372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:17.764023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:16:17.764319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:17.765663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:17.767780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:17.773691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 600/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:16:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:16:17.779603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:17.856542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:17.877063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa6dd5-5ae9-4025-ab1b-815eeec990eb None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:17.909467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:17.910078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:17.910350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:17.910612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] PATH_INFO: `/v3/users/01cf91e2257a41718ef9051b804d7bb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:17.911731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] RBAC: Authorizing `identity:get_user(user_id=01cf91e2257a41718ef9051b804d7bb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:17.913140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:17.918710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 597/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Mon Aug 8 00:16:17 2022] GET /identity/v3/users/01cf91e2257a41718ef9051b804d7bb0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:17.921069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:18.017574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c80430-142c-47a7-a8ef-e5c60ed1fa7c None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:18.049601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:18.050183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:18.050459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:18.050726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:18.111245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:18.114154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:18.115600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:18.209731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 601/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:16:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:16:18.216202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:18.311843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5f7f-d65b-4355-b1e8-7e79cfcda53f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:18.345585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c852ed-faf9-4f53-a556-4cdb59b7e9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:18.346193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c852ed-faf9-4f53-a556-4cdb59b7e9f9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:18.346467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c852ed-faf9-4f53-a556-4cdb59b7e9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:18.346734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c852ed-faf9-4f53-a556-4cdb59b7e9f9 None admin] PATH_INFO: `/v3/projects/2a9460fd27ca4e55b47db9592bab6c22` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:18.347821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c852ed-faf9-4f53-a556-4cdb59b7e9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2a9460fd27ca4e55b47db9592bab6c22)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:18.349237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c852ed-faf9-4f53-a556-4cdb59b7e9f9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:18.350826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 598/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Mon Aug 8 00:16:18 2022] GET /identity/v3/projects/2a9460fd27ca4e55b47db9592bab6c22 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:18.357643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:16:18.448333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:18.452014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386fc0a-72fe-4934-8054-7b5facea896a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:16:18.485861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:16:18.486382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:16:18.486656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:16:18.486921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:16:18.487651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=2a9460fd27ca4e55b47db9592bab6c22, user.id=01cf91e2257a41718ef9051b804d7bb0) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:16:18.487952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:16:18.489346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191747d7-cb50-4636-ae2a-f371e1358d49 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:16:18.495406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 602/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:16:18 2022] GET /identity/v3/role_assignments?user.id=01cf91e2257a41718ef9051b804d7bb0&scope.project.id=2a9460fd27ca4e55b47db9592bab6c22&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:16:28.341482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=93 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:16:28.496213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=19 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:31.706556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f050ff-9fdf-4189-b6a0-853ac5567177 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:31.707068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f050ff-9fdf-4189-b6a0-853ac5567177 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:31.707068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f050ff-9fdf-4189-b6a0-853ac5567177 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:31.708456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 599/1201] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:31.714441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:31.714844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:31.714974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:31.732754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:31.733910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:31.797378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 603/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:31.822885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f050ff-9fdf-4189-b6a0-853ac5567177 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:31.923595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f050ff-9fdf-4189-b6a0-853ac5567177 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:31.957910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:31.958550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:31.958829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:31.959101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:31.969082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:31.970537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:31.984651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:31.993879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:31.994891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:31.994891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:31.994891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:31.994891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 600/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:31.995170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:32.092169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dce6fa-675f-487f-928c-a1a5088f209e None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:32.120379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:32.125946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:32.126543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:32.126824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:32.127094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:32.127748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:32.128045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:32.129394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac829e-68d8-4d1a-907a-4ac132c5fd46 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:32.140303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 604/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:32.145902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:32.199162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:32.245144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215c43e-e4f6-44a3-92eb-1e7dd34a321a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:32.277809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16bea11-0439-44f5-9c89-cd0dce8aa4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:32.278400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bea11-0439-44f5-9c89-cd0dce8aa4de None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:32.278674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bea11-0439-44f5-9c89-cd0dce8aa4de None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:32.278943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16bea11-0439-44f5-9c89-cd0dce8aa4de None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:32.279610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16bea11-0439-44f5-9c89-cd0dce8aa4de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:32.281024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16bea11-0439-44f5-9c89-cd0dce8aa4de None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:32.392460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 601/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Mon Aug 8 00:17:32 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:17:34.443005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f0b07-622f-4bb5-baf4-e38d8bad04bc None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:34.443573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f0b07-622f-4bb5-baf4-e38d8bad04bc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:34.443573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f0b07-622f-4bb5-baf4-e38d8bad04bc None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:34.445190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 605/1206] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:34.450859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:34.451311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:34.451537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:34.470257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:34.471449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:34.539331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 602/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:34.562381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f0b07-622f-4bb5-baf4-e38d8bad04bc None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:34.665497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f0b07-622f-4bb5-baf4-e38d8bad04bc None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:34.698676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:34.699248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:34.699600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:34.699869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:34.711047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:17:34.715809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:34.716807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:34.717944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 606/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:17:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:34.722893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:34.762234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:34.820312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552afc2b-5f36-4407-aa01-7d16126770e0 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:34.852469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:34.857526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:34.857526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:34.857526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:34.857526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:34.857526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:34.857526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:34.863626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 603/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:17:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:34.874170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:34.922937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:34.971885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e64be-b120-4ca1-b6c2-9d9642c1148f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:35.004575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f512002-6791-4662-b633-97def0a00645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:35.005190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f512002-6791-4662-b633-97def0a00645 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:35.005468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f512002-6791-4662-b633-97def0a00645 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:35.005738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f512002-6791-4662-b633-97def0a00645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:35.016640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f512002-6791-4662-b633-97def0a00645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:35.018625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f512002-6791-4662-b633-97def0a00645 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-9f512002-6791-4662-b633-97def0a00645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:35.031932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:35.033046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:35.034314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:35.034314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:35.034314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:35.034314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 607/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:35.038857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:35.133402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:35.136252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e3f2d-ee51-4448-9392-0916ffb6f66d None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:35.168864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:35.169460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:35.169733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:35.170000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:35.170624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:35.170916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:35.172310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:35.183256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 604/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:17:35.188952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f512002-6791-4662-b633-97def0a00645 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:35.291095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f512002-6791-4662-b633-97def0a00645 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:35.292602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:35.324594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:35.325206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:35.325480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:35.325745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] PATH_INFO: `/v3/users/placement` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:35.332952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:35.334363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:17:35.345472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:35.346602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:35.348040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 608/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:35.353203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:35.450623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fbac99-ed68-4f1d-b3f4-bee44ef92509 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:35.483417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:35.484016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:35.484291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:35.484558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:35.485271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:35.485570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:35.487026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:35.504283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 605/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 00:17:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:35.510346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:35.609949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8881e0-c370-4536-93b8-d0e5ab3edebe None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:35.644044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:35.644642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:35.644945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:35.645221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:35.676325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:35.713775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:35.715214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:35.812592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 609/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:17:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:17:35.818687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:35.838392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:35.917376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117a6fd-86ec-4a13-9b83-2bd3dde28865 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:35.954464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:35.955087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:35.955395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:35.955670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:35.965582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:35.967009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:35.980903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:35.982041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:35.983391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:35.983391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:35.983391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:35.983391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 606/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:35.987807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:36.047746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:36.085732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cddaac-e7f4-431a-9abe-bab9f2f710c1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:36.118860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:36.119483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:36.119757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:36.120031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:36.120655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:36.120950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:36.122378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:36.134003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 610/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:36.139522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:36.207196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:36.235914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a7f9-2d23-473b-bf90-9dc52aa15052 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:36.267827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:36.268457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:36.268718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:36.268990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:36.276093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:36.280836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:17:36.288790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:36.289845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:36.295396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 607/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:36.296049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:36.393138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefaf5f3-f59d-40b9-af22-e9cc2e7fd46a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:36.414790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:36.425776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:36.426372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:36.426646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:36.426914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:36.427622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:36.427920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:36.430064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142d655-7286-4668-bad3-959a3b1c9bb4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:36.441476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 611/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:17:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:17:36.448178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:36.545847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967eb99d-e244-4fa2-abe6-9ad3593c6f0d None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:36.574196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:36.580008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:36.580613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:36.580889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:36.581157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:36.581862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=21c1437c38d743c585ce98f7a3e96627) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:36.582161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:36.583605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb574333-4fd0-45ba-886b-a2d56e5ddcb1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:36.588895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 608/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:17:36 2022] GET /identity/v3/role_assignments?user.id=21c1437c38d743c585ce98f7a3e96627&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:38.581109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db2e9-6ff3-4703-aaea-5fb924df533c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:38.581696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db2e9-6ff3-4703-aaea-5fb924df533c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:38.581696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db2e9-6ff3-4703-aaea-5fb924df533c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:38.582981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 612/1220] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:38.588485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:38.588883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:38.589032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:38.607130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:38.608396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:38.673985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 609/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:38.697281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db2e9-6ff3-4703-aaea-5fb924df533c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:38.797891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db2e9-6ff3-4703-aaea-5fb924df533c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:38.831900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:38.832507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:38.832787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:38.833056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:38.843027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:17:38.847657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:38.851835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:38.852870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 613/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:17:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:17:38.855314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:38.951783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b8dea-a7f4-4ccd-92b6-89671ee4138d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:38.975124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:38.986011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:38.986606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:38.986881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:38.987150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:38.987830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:38.988125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:38.989469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:38.997059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 610/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:17:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:39.002791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.046459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:39.101267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687d682-b51c-4f80-ad0f-bc78436d36b5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:39.133402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:39.134015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:39.134291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:39.134565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:39.144050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:39.145533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:39.160745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:39.164219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:39.165358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:39.165358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:39.165358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:39.165358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 614/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:39.168538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.266716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e417535-be63-4875-bb61-9432dc7b6cd7 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:39.301986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:39.302601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:39.302885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:39.303159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:39.303853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:39.304153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:39.305514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:39.316867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 611/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:39.323861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.429700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:39.436195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5983ef2-eed9-4d7d-9634-f1fa0e9c119d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:39.469148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:39.469790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:39.470071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:39.470347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:39.477671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:39.479097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:17:39.489809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:39.490979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:39.492078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 615/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:39.496957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.544261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:39.595779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054e6e4-17e7-4217-8bec-982c7770afa0 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:39.628786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:39.629402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:39.629684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:39.629951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:39.630663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:39.630965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:39.633211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:39.650392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 612/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:39.656153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.753402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77879-ffa3-4fde-b729-bc92fd60d3c3 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:39.786264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:39.787736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:39.787736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:39.787736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:39.797797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:39.799309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:39.813559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:39.818595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:39.820345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:39.820345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:39.820345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:39.820345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 616/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:39.821904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.918520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18d54-e7e5-4523-a27b-77a9236de736 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:39.920783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:39.951885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:39.952472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:39.952745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:39.953009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:39.953651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:39.953932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:39.955437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:39.966390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 613/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:39.975350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:39.981620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:40.073230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd7d78-5d50-4c83-81cc-d82ba4af37f1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:40.106062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:40.112918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:40.112918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:40.112918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:40.113900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:40.115385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:17:40.126712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:40.127795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:40.129056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 617/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:17:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:40.133772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:40.231921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246ced6-35ba-4f8c-931a-1da671bc5a71 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:40.264461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:40.265256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:40.265363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:40.265630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:40.266308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:40.266606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:40.268035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78fe03-7bda-41ca-8e70-dbedcea77ac8 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:40.279781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 614/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:17:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:40.285829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:40.351951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:40.385691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b43dc7-d799-478d-b312-da8273951dfa None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:40.418417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62291c08-d6ea-41e0-852b-94fd05a41c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:40.419056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62291c08-d6ea-41e0-852b-94fd05a41c80 None admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:40.419360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62291c08-d6ea-41e0-852b-94fd05a41c80 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:40.419724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62291c08-d6ea-41e0-852b-94fd05a41c80 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/21c1437c38d743c585ce98f7a3e96627/roles/1c7df90ed0744af7b4b5c8e433fff9f0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:40.433854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62291c08-d6ea-41e0-852b-94fd05a41c80 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=21c1437c38d743c585ce98f7a3e96627, role_id=1c7df90ed0744af7b4b5c8e433fff9f0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:40.435281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62291c08-d6ea-41e0-852b-94fd05a41c80 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:40.498479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 618/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:17:40 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/21c1437c38d743c585ce98f7a3e96627/roles/1c7df90ed0744af7b4b5c8e433fff9f0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:17:42.477914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f959755-2def-437c-88fb-8d6fb4ad76fc None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:42.478467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f959755-2def-437c-88fb-8d6fb4ad76fc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:42.478467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f959755-2def-437c-88fb-8d6fb4ad76fc None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:42.479808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 615/1233] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:42.485034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:42.485435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:42.485578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:42.504416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:42.505584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:42.571187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 619/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:42.594267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f959755-2def-437c-88fb-8d6fb4ad76fc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:42.620771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:42.695693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f959755-2def-437c-88fb-8d6fb4ad76fc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:42.730451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:42.730997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:42.731319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:42.731593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:42.741565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:17:42.746225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:42.747369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:42.748256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 616/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Mon Aug 8 00:17:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:42.753405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:42.797749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:42.847820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef8612-8af3-4845-baf2-56a931b447a6 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:42.885535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:42.886143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:42.886420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:42.886692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:42.887383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] RBAC: Adding query filter params (name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:42.888709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:42.889060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:42.897520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 620/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Mon Aug 8 00:17:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:42.903363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:42.987099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:43.001655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd0e96-bb59-43da-9b14-67308b744bb3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:43.035590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:43.036200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:43.036472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:43.036739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:43.046256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:43.047748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:43.061692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:43.062822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:43.063784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:43.063784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:43.063784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:43.063784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 617/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:43.069072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:43.168095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762a2c-fc4b-41d5-8a5c-a11f9d2ffd2a None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:43.176732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:43.200903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:43.201493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:43.201803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:43.202072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:43.202695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:43.202992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:43.204378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:43.216828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 621/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:43.222569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:43.322880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a86f82-9b89-4967-9fd0-b4bbeb43310c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:43.356069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:43.356718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:43.357323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:43.357606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:43.360319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:43.365260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:43.367252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:17:43.378015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:43.379142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:43.380362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 618/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:43.385800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:43.482945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d7c0f-560c-4cba-a102-b6de5769c7e4 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:43.517422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:43.518068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:43.518292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:43.518565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:43.519335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:43.519643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:43.521069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:43.538090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 622/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 00:17:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:43.543972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:43.643827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd013d69-5a03-4c5c-bd5c-69055ddf96a1 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:43.678695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:43.679414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:43.679693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:43.679961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:43.739381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:43.746325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:43.747793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:43.846928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 619/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:17:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:17:43.853815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:43.880712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:43.948922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274c9d9-561d-499d-8d7a-96ce1a76bbae None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:43.982422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:43.983046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:43.983353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:43.983629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:43.994134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:43.995637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:44.010710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:44.011855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:44.012779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:44.012779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:44.012779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:44.012779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 623/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:44.017697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:44.112452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:44.118902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22da32e-2bbf-423b-a644-fc02291f4a1b None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:44.151523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:44.152120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:44.152397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:44.152668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:44.153302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:44.153600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:44.154969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:44.166266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 620/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:44.172210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:44.248444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:44.268617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48a6ec-c02f-4122-b5a9-5a83591dfe13 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:44.301461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:44.302076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:44.302351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:44.302619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:44.309689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:44.311154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:17:44.322399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:44.323489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:44.324448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 624/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:17:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:44.329644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:44.429257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49060e4-3b39-4b9d-94b7-d38a256ef91c None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:44.463029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:44.463691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:44.463965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:44.464231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:44.464906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:44.465203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:44.466619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46ec86-2a65-455c-b66d-8cb5cf4a44b9 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:44.478997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 621/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:17:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:44.485173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:44.585623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf834b0-4d4f-4934-b2d8-55670d618c4e None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:44.618219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:44.618809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:44.619085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:44.619561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:44.620280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] RBAC: Adding query filter params (role.id=1c7df90ed0744af7b4b5c8e433fff9f0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=21c1437c38d743c585ce98f7a3e96627) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:44.620583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:44.621986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a70170-9697-410c-a957-a5f9705d1916 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:44.627681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 625/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:17:44 2022] GET /identity/v3/role_assignments?user.id=21c1437c38d743c585ce98f7a3e96627&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=1c7df90ed0744af7b4b5c8e433fff9f0 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:46.715939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194614b0-aada-4a88-b798-085b67687b31 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:46.716514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194614b0-aada-4a88-b798-085b67687b31 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:46.716514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194614b0-aada-4a88-b798-085b67687b31 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:46.717924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 622/1247] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:46.723132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:46.723635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:46.723716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:46.730914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:46.743047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:46.744359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:46.814528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 626/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:46.839830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194614b0-aada-4a88-b798-085b67687b31 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:46.859612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:46.949645 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194614b0-aada-4a88-b798-085b67687b31 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:46.983493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:46.984118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:46.984400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:46.984675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:46.994848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:17:46.999512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:47.000686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:47.002023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 623/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:17:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:47.007338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:47.104949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22410e4-d75a-4b2a-80c9-967564178a4a None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:47.134440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:47.139608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:47.140221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:47.140504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:47.140781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:47.141443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:47.141750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:47.143118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:47.152588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 627/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:17:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:47.158406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:47.235806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:47.258759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90512522-1c59-417e-8317-dd1224e2c509 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:47.291105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:47.291749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:47.292027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:47.292296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:47.301763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:47.303784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:47.317753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:47.319948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:47.321154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:47.321154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:47.321154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:47.321154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 624/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:47.325440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:47.422404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf9a31-3e25-4480-95a1-7cfc531df517 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:47.457918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:47.458516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:47.458792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:47.459094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:47.459850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:47.460151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:47.461525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:47.473411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 628/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:47.479365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:47.578474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1610b-5549-440f-8bfc-335969aa451f None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:47.604279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:47.611805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:47.612429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:47.612737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:47.613009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] PATH_INFO: `/v3/users/placement` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:47.620844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:47.622284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:17:47.633263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:47.641856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:47.643083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 625/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:47.643850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:47.696279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:47.739506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7bdb8-0f3a-47ee-97ce-656ddfcf96e1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:47.772433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:47.773029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:47.773306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:47.773577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:47.774289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:47.774592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:47.776031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:47.793227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 629/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 00:17:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:47.799109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:47.897884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450754d-1bb9-4ac5-8d79-3f55b7b9225a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:47.929980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:47.930564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:47.930837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:47.931100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:47.983844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:47.995841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:47.997271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:48.094089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 626/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:17:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:17:48.100723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:48.198610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b77d-624d-4f67-8f2b-31de880d5830 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:48.212179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:48.232626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:48.233238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:48.233512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:48.233779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:48.243436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:48.244853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:48.259073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:48.260240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:48.261208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:48.261208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:48.261208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:48.261208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 630/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:48.266124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:48.352995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:48.364412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0764f-e6e6-4d97-8e5a-381bcf5a86f3 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:48.398347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:48.398942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:48.399247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:48.399501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:48.400123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:48.402139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:48.402139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:48.412593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 627/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:48.418331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:48.515997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac80923-ef29-40fd-8306-390270f0bb2f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:48.549114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:48.549727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:48.549999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:48.550265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:48.557293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:48.558731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:17:48.569811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:48.571188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:48.572361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 631/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:17:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:48.576915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:48.676324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91dddaa-e632-474e-aa0d-fd9020befba2 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:48.709640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:48.710367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:48.710643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:48.710909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:48.711632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:48.711932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:48.713356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d7500-8d6f-40da-88b0-6e00f749777c None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:48.725876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 628/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:17:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:48.732121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:48.758410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:48.828978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647692f-f83a-4932-b722-17bf0d7df875 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:48.861590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:48.862176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:48.862452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:48.862722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:48.863457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=21c1437c38d743c585ce98f7a3e96627) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:48.863759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:48.865203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3715a3a-0b01-45fe-8c0c-54058af9716b None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:48.870525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 632/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:17:48 2022] GET /identity/v3/role_assignments?user.id=21c1437c38d743c585ce98f7a3e96627&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:50.845825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d1ab-4b87-42f3-ab02-3db130f89ab8 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:50.846383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d1ab-4b87-42f3-ab02-3db130f89ab8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:50.846383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9d1ab-4b87-42f3-ab02-3db130f89ab8 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:50.847726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 629/1261] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:50.853670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:50.854093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:50.854243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:50.873430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:50.874617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:50.941366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 633/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:50.964072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d1ab-4b87-42f3-ab02-3db130f89ab8 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:50.975292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:51.062704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9d1ab-4b87-42f3-ab02-3db130f89ab8 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:51.096420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:51.097041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:51.097319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:51.097589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:51.107931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:17:51.112614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:51.117588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:51.118718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 630/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:17:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:51.119919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:51.142054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:51.220429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886db4cd-3e05-45d8-bb0d-21662a91c727 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:51.253132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:51.253753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:51.254028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:51.254299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:51.254953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:51.255282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:51.257163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:51.264836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 634/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:17:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:51.270768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:51.337921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:51.369795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762163a-42e7-4c2e-9542-bd9fe22912d9 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:51.403321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:51.403940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:51.404218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:51.404488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:51.414864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:51.416354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:51.429874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:51.431059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:51.432212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:51.432212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:51.432212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:51.432212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 631/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:51.436777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:51.517984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:51.536824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b9483-5a12-4ac6-813e-6e9d2fe0064d None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:51.568009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:51.569461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:51.569729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:51.570015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:51.570618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:51.570916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:51.572288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:51.583458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 635/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:51.590191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:51.689133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5ffa6-a616-4b4d-bc04-054d4e8a9424 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:51.709660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:51.725555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:51.726179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:51.726455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:51.726723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] PATH_INFO: `/v3/users/placement` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:51.734303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:51.735779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:17:51.746890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:51.748095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:51.749412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 632/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:51.753986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:51.852036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae95709-13c1-4a5a-bd40-d2568a32c7cd None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:51.886016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:51.886600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:51.886877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:51.887145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:51.887889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:51.888195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:51.889617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:51.898323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:51.906597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 636/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 00:17:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:51.912508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:52.012784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f537db4-53db-4080-928a-4cdfdb03325a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:52.045901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:52.046548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:52.046831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:52.047131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:52.057911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:52.059389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:52.073318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:52.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:52.075789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:52.075789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:52.075789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:52.075789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 633/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:52.080248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:52.180214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1480cef-e5d0-46de-a57b-996b6d6ce598 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:52.213307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:52.213919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:52.214201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:52.214479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:52.215119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:52.215461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:52.216872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:52.228327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 637/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:52.234066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:52.265867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:52.333356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f461b7-bd76-4b9b-a047-780aba7a65f5 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:52.369229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:52.369847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:52.370123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:52.370392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:52.376929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:52.378384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:17:52.389959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:52.390970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:52.391995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 634/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:17:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:52.397615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:52.455395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:52.495084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f42783-c2fc-4de9-89a4-9d81d1db9798 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:52.528418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:52.529014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:52.529290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:52.529560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:52.530230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:52.530529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:52.531967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffa477-bb3d-4b53-9a9d-71d2db7b3f61 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:52.544022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 638/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:17:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:52.549960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:52.638357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:52.647160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa499d5-1124-4e26-8b07-ad8ada737c80 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:52.681063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31a620-12bc-4326-9d2d-138725c7a8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:52.681746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31a620-12bc-4326-9d2d-138725c7a8f4 None admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:52.682024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31a620-12bc-4326-9d2d-138725c7a8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:52.682297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31a620-12bc-4326-9d2d-138725c7a8f4 None admin] PATH_INFO: `/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/21c1437c38d743c585ce98f7a3e96627/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:52.684203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31a620-12bc-4326-9d2d-138725c7a8f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user_id=21c1437c38d743c585ce98f7a3e96627, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:52.685619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31a620-12bc-4326-9d2d-138725c7a8f4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:52.746275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 635/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Mon Aug 8 00:17:52 2022] PUT /identity/v3/projects/1d7e3c85c1c141d1965b1c8f0a5cbe21/users/21c1437c38d743c585ce98f7a3e96627/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:17:54.758479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b3c4d-f4f6-4944-b210-61d771dc7084 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:54.759036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b3c4d-f4f6-4944-b210-61d771dc7084 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:54.759036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b3c4d-f4f6-4944-b210-61d771dc7084 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:54.760380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 639/1274] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:54.765635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:54.766085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:54.766181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:54.784836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:54.786091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:54.859576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 636/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:54.883368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b3c4d-f4f6-4944-b210-61d771dc7084 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:54.983948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b3c4d-f4f6-4944-b210-61d771dc7084 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.001036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:55.017479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.018096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.018375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.018648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:55.028922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 00:17:55.033590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:55.034749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:55.035997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 640/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Mon Aug 8 00:17:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:55.041007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:55.123291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:55.141564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594c4c4-e836-4de8-8083-b57efb99cad6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.174028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.174624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.174906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.175178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:55.175871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:55.176173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:55.178236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:55.186096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 637/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Mon Aug 8 00:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:55.191880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:55.292270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b82caad-f9a3-464b-92f6-3c4bbf98669f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.326606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.327251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.327555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.327862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:55.337790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:55.339664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:55.353751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:55.354899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:55.355902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:55.355902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:55.355902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:55.355902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 641/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:55.360656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:55.461462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb6236-503f-472b-a082-505be8dde60a None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.496531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.497135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.497416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.497684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:55.498315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:55.498612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:55.500038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:55.511798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 638/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:55.517643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:55.548689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:55.618011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59509a30-354d-4cdd-8f00-348bb5c1b5f1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.650346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.650985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.651284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.651556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:55.659073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:55.660543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 00:17:55.672118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:55.673226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:55.674452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 642/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:55.679771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:55.683499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:55.777301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40763cc-8281-440f-83f7-2223b02ebb42 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.811691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.812223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.812497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.812763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:55.813516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:55.813813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:55.815251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:55.834127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 639/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Mon Aug 8 00:17:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:55.840114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:55.929541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:55.938596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa4c9e-6710-41b6-91d3-4e40e33d1080 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:55.971800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:55.972400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:55.972683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:55.972952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:56.038388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:56.039863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:56.125192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:56.135216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 643/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Mon Aug 8 00:17:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 00:17:56.141251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:56.201113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:56.238130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49896-25d6-49be-941c-7168b241f6dd None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:56.270098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:56.270727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:56.271022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:56.271328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:56.281423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:56.282874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 00:17:56.298076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:56.299306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 00:17:56.300575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 00:17:56.300575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 00:17:56.300575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:56.300575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 640/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:17:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:56.305588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:56.402758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91c8223-b4d0-4770-9c94-50d40f6974b1 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:56.436262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:56.436854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:56.437134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:56.437405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:56.438031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:56.438334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:56.439740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:56.454707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 644/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Mon Aug 8 00:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:56.460400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:56.559862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c285189-887f-4f66-bdc9-a16350e99ec4 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:56.573581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:56.593324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:56.593941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:56.594218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:56.594490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:56.601568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:56.603015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 00:17:56.614038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:56.615309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: ERROR keystone.server.flask.application Aug 08 00:17:56.616270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 641/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Mon Aug 8 00:17:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:56.621245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:56.671518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:56.719256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c268dfd-818b-4d78-98b7-01a99f74a7ab None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:56.753318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:56.754047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:56.754341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:56.754602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:56.755324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:56.755641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:56.757228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f5412-539b-48fb-b318-ae64c593d0aa None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:56.769008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 645/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Mon Aug 8 00:17:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:56.775714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:56.872974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839df2-aa87-4f1e-ba6e-c250fbaa0ae6 None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:56.906018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:56.906615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:56.906882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:56.907149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:56.907901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] RBAC: Adding query filter params (role.id=ad68083fe37d48cba8726cfd30e43fb0, scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21, user.id=21c1437c38d743c585ce98f7a3e96627) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:56.908206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:56.909633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22b5a-e064-43c0-9dcb-58aa63b19a56 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:56.915964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 642/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Mon Aug 8 00:17:56 2022] GET /identity/v3/role_assignments?user.id=21c1437c38d743c585ce98f7a3e96627&scope.project.id=1d7e3c85c1c141d1965b1c8f0a5cbe21&role.id=ad68083fe37d48cba8726cfd30e43fb0 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:58.944841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591f3dd-8180-46ce-9ef1-fd10893e53ed None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:58.945350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591f3dd-8180-46ce-9ef1-fd10893e53ed None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:58.945350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591f3dd-8180-46ce-9ef1-fd10893e53ed None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:58.946677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 646/1288] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:17:58.951939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:58.952340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:58.952481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:58.970990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:17:58.972262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:17:59.001761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:59.037854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 643/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:17:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:17:59.060024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591f3dd-8180-46ce-9ef1-fd10893e53ed None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:59.159138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591f3dd-8180-46ce-9ef1-fd10893e53ed None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:59.172039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:17:59.197729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:59.198302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:59.198581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:59.198856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] PATH_INFO: `/v3/services/placement` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:59.199531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:59.200954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:17:59.211549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:17:59.212868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:17:59.213939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:17:59.215412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 647/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:17:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:17:59.219053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:59.327092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63bf8b7-d64b-495c-b745-f5be2f6411ef None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:59.372824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:59.373459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:59.373736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:59.374005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:59.374684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] RBAC: Adding query filter params (name=placement) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:59.374988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:59.376464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936fd6e-d5db-4350-9b1d-cbe76046b46e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:59.382505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 644/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 00:17:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:17:59.388465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:17:59.503318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8bc0a5-afaa-4eb7-86cb-191c642f691f None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:17:59.537945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:17:59.538712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:17:59.538998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:17:59.539291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] PATH_INFO: `/v3/services` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:17:59.539916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] RBAC: Adding query filter params (type=placement) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:17:59.540225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:17:59.541568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52116b78-293c-4b1c-8dea-e325fadb590c None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:17:59.546114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 648/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 00:17:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:18:01.566557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edd861-7df2-4485-971f-8f554f06e8ca None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:01.567169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edd861-7df2-4485-971f-8f554f06e8ca None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:01.567169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edd861-7df2-4485-971f-8f554f06e8ca None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:01.569261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 645/1293] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:18:01.574536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc78711-c904-491b-9c09-33c59e2cf21c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:01.574941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc78711-c904-491b-9c09-33c59e2cf21c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:01.575075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc78711-c904-491b-9c09-33c59e2cf21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:01.593429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-dfc78711-c904-491b-9c09-33c59e2cf21c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:18:01.594654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dfc78711-c904-491b-9c09-33c59e2cf21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:18:01.624257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:01.665819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 649/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:18:01.688256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edd861-7df2-4485-971f-8f554f06e8ca None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:01.721381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:01.792098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edd861-7df2-4485-971f-8f554f06e8ca None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:01.826303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f721c-fd24-4390-a556-eef1a41b374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:01.826842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f721c-fd24-4390-a556-eef1a41b374d None admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:01.827125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f721c-fd24-4390-a556-eef1a41b374d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:01.827431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f721c-fd24-4390-a556-eef1a41b374d None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:01.828062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f721c-fd24-4390-a556-eef1a41b374d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:01.829425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f721c-fd24-4390-a556-eef1a41b374d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:01.911388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 646/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Mon Aug 8 00:18:01 2022] POST /identity/v3/services => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:18:03.889692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da401ab-7ce0-4c9b-91c9-56d9fd2f27da None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:03.890241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da401ab-7ce0-4c9b-91c9-56d9fd2f27da None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:03.890241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da401ab-7ce0-4c9b-91c9-56d9fd2f27da None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:03.891741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 650/1296] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:18:03.897627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:03.898078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:03.898173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:03.915893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:18:03.917058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:18:03.989471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 647/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:18:03 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 00:18:04.012033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da401ab-7ce0-4c9b-91c9-56d9fd2f27da None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:04.110381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da401ab-7ce0-4c9b-91c9-56d9fd2f27da None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:04.143707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:04.144322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:04.144598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:04.144868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:04.145440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:04.146811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:18:04.155190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:18:04.156368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:18:04.157388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 651/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:18:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 00:18:04.162322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:04.181151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:04.259836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66266ca-f46c-4b0c-9775-66a5c126d307 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:04.291826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:04.292412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:04.292687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:04.293134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:04.293749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] RBAC: Adding query filter params (name=placement) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:18:04.294045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:04.295423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:04.300560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 648/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 00:18:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:18:04.307075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:04.338629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:04.405491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefcd7e-2af5-419b-9847-43b1c399aea5 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:04.438073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:04.438658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:04.438933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:04.439206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:04.439915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] RBAC: Adding query filter params (interface=public, service_id=4933f986f83e46fcb4c4d1df017ff874, region_id=RegionOne) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:18:04.440216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:04.441549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b077d21-ef74-4061-a7a4-0e5085ab310f None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:04.446898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 652/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Mon Aug 8 00:18:04 2022] GET /identity/v3/endpoints?service_id=4933f986f83e46fcb4c4d1df017ff874&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:18:04.452611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:04.532365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:04.550087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cde38a-faed-4ba1-b817-69afa674ab7e None admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:04.584915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9f263-e853-4df9-9909-20d08828b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:04.585509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9f263-e853-4df9-9909-20d08828b32d None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:04.585814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9f263-e853-4df9-9909-20d08828b32d None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:04.586083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9f263-e853-4df9-9909-20d08828b32d None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:04.586672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9f263-e853-4df9-9909-20d08828b32d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:04.588049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9f263-e853-4df9-9909-20d08828b32d None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:04.594534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 649/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Mon Aug 8 00:18:04 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:18:06.586206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795202e-dac1-4a88-8ad2-b08e92edd36b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:06.586766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795202e-dac1-4a88-8ad2-b08e92edd36b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:06.586766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795202e-dac1-4a88-8ad2-b08e92edd36b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:06.588125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 653/1302] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:18:06.593318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:06.593748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:06.593842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:06.612029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:18:06.613207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:18:06.679622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 650/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:18:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 00:18:06.702888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795202e-dac1-4a88-8ad2-b08e92edd36b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:06.795486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:06.802062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795202e-dac1-4a88-8ad2-b08e92edd36b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:06.835467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:06.836112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:06.836435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:06.836654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:06.837221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:06.838588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 00:18:06.847086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 00:18:06.848278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application with Lock( Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self._enter() Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application return self.creator() Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application created_value = creator( Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: ERROR keystone.server.flask.application Aug 08 00:18:06.849341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 654/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Mon Aug 8 00:18:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 00:18:06.854468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:06.929821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:06.952149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba74d53-afa0-46e3-ab9a-ce317e87328d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:06.985928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:06.986517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:06.986787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:06.987050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] PATH_INFO: `/v3/services` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:06.987689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:18:06.987986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:06.989545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c700c58-2d2e-4033-9637-ace7296e7053 None admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:06.994650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 651/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Mon Aug 8 00:18:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:18:07.000433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:18:07.100805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f113f-a8f3-47fc-ac8d-1c5b14774c24 None admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:18:07.133564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900be408-c230-47ad-a4e0-6828e3c03186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:18:07.134176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900be408-c230-47ad-a4e0-6828e3c03186 None admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:18:07.134461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900be408-c230-47ad-a4e0-6828e3c03186 None admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:18:07.134733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900be408-c230-47ad-a4e0-6828e3c03186 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:18:07.135425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900be408-c230-47ad-a4e0-6828e3c03186 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:18:07.136811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900be408-c230-47ad-a4e0-6828e3c03186 None admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:18:07.277502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 655/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:18:06 2022] POST /identity/v3/endpoints => generated 331 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:18:16.991702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=30 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:18:17.273016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:13.782677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785a99-ba31-4879-b900-02fb2b049bd3 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:13.783266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785a99-ba31-4879-b900-02fb2b049bd3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:13.783266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a785a99-ba31-4879-b900-02fb2b049bd3 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:13.784924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 652/1307] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:24:13.790102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f2bdf-217c-490e-a2dd-8d58507293ca None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:13.790577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f2bdf-217c-490e-a2dd-8d58507293ca None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:13.790674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f2bdf-217c-490e-a2dd-8d58507293ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:13.959328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-fb3f2bdf-217c-490e-a2dd-8d58507293ca None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:13.960997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fb3f2bdf-217c-490e-a2dd-8d58507293ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:14.107085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 656/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:14.158370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99212302-7670-47ec-981d-81f2935b1d67 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:14.158650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99212302-7670-47ec-981d-81f2935b1d67 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:14.158949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99212302-7670-47ec-981d-81f2935b1d67 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:14.160822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 653/1309] 10.222.0.15 () {54 vars in 965 bytes} [Mon Aug 8 00:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:14.172403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19bb38-2ca1-4082-b724-6229e4567555 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:14.172613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19bb38-2ca1-4082-b724-6229e4567555 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:14.173092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19bb38-2ca1-4082-b724-6229e4567555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:14.269787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0b19bb38-2ca1-4082-b724-6229e4567555 None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:14.271573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0b19bb38-2ca1-4082-b724-6229e4567555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:14.308652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:14.369903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 657/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Aug 8 00:24:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:24:14.380344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99212302-7670-47ec-981d-81f2935b1d67 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:14.382203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-99212302-7670-47ec-981d-81f2935b1d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:14.495754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99212302-7670-47ec-981d-81f2935b1d67 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:14.572737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:14.573343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:14.573617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:14.573879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:14.575508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:14.631817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:14.694305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:14.695904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b5a9-c8ca-4035-8c84-567861dfe10a service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:14.715353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 654/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:17.050178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055637-65c7-4e8a-8675-2457b59ce8d7 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:17.050673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055637-65c7-4e8a-8675-2457b59ce8d7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:17.051036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60055637-65c7-4e8a-8675-2457b59ce8d7 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:17.052574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 658/1312] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:17.057500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973b236-e3c1-4897-9b50-c2392a8934d1 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:17.057928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973b236-e3c1-4897-9b50-c2392a8934d1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:17.058148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973b236-e3c1-4897-9b50-c2392a8934d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:17.134899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c973b236-e3c1-4897-9b50-c2392a8934d1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:17.136386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c973b236-e3c1-4897-9b50-c2392a8934d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:17.247241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 655/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:17.287156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055637-65c7-4e8a-8675-2457b59ce8d7 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:17.289073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-60055637-65c7-4e8a-8675-2457b59ce8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:17.391307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60055637-65c7-4e8a-8675-2457b59ce8d7 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:17.461947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:17.464529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:17.465131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:17.465407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:17.465727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:17.467709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:17.586006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:17.587513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a1d1c-c996-40c3-9999-6ab72f7bee2f service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:17.607805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 659/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:20.084193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d3aa9-6f53-488b-adf5-d7c055f81e69 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:20.084745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d3aa9-6f53-488b-adf5-d7c055f81e69 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:20.084745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d3aa9-6f53-488b-adf5-d7c055f81e69 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:20.085996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 656/1315] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:20.091022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8666e4a-7f67-4824-8249-8a57814affed None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:20.091487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8666e4a-7f67-4824-8249-8a57814affed None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:20.091582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8666e4a-7f67-4824-8249-8a57814affed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:20.154758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a8666e4a-7f67-4824-8249-8a57814affed None None] MFA Rules not processed for user `49a016ba549949ef81523de0e9a64871`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:20.156009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a8666e4a-7f67-4824-8249-8a57814affed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:20.206461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:20.249418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 660/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:20.290236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d3aa9-6f53-488b-adf5-d7c055f81e69 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:20.291849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-584d3aa9-6f53-488b-adf5-d7c055f81e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:20.301440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:20.395543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d3aa9-6f53-488b-adf5-d7c055f81e69 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:20.467861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:20.468462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:20.468762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:20.469030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:20.470616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:20.531765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:20.577203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:20.578636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab248cd2-9791-46b8-bfdb-547ce89a55b2 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:20.598089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 657/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:23.400566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532eb19-07c5-4391-a6a5-c1a3a646ee4d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:23.401080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532eb19-07c5-4391-a6a5-c1a3a646ee4d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:23.401080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532eb19-07c5-4391-a6a5-c1a3a646ee4d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:23.402369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 661/1318] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:23.407574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba6d2-cb1a-48d7-9f6b-5cb6d42c5212 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:23.407984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba6d2-cb1a-48d7-9f6b-5cb6d42c5212 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:23.408126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba6d2-cb1a-48d7-9f6b-5cb6d42c5212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:23.474479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b6fba6d2-cb1a-48d7-9f6b-5cb6d42c5212 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:23.475699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b6fba6d2-cb1a-48d7-9f6b-5cb6d42c5212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:23.569796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 658/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:23.609308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532eb19-07c5-4391-a6a5-c1a3a646ee4d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:23.610889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e532eb19-07c5-4391-a6a5-c1a3a646ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:23.713806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532eb19-07c5-4391-a6a5-c1a3a646ee4d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:23.783468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:23.784185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:23.784461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:23.784731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:23.786312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:23.793400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:23.890344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:23.891798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e476ed1-602e-457e-b651-a1ccffae572f service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:23.910239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 662/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:26.882909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13850fcd-5e58-4d51-8bb4-cfeef2e673c6 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:26.883465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13850fcd-5e58-4d51-8bb4-cfeef2e673c6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:26.883465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13850fcd-5e58-4d51-8bb4-cfeef2e673c6 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:26.884809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 659/1321] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:26.889969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb324-b509-423b-876c-b6b3447cd518 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:26.890412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb324-b509-423b-876c-b6b3447cd518 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:26.890540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcb324-b509-423b-876c-b6b3447cd518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:26.956606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-85bcb324-b509-423b-876c-b6b3447cd518 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:26.957719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-85bcb324-b509-423b-876c-b6b3447cd518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:27.013824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:27.054066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 663/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:27.091986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13850fcd-5e58-4d51-8bb4-cfeef2e673c6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:27.093628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-13850fcd-5e58-4d51-8bb4-cfeef2e673c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:27.115597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:27.205139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13850fcd-5e58-4d51-8bb4-cfeef2e673c6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:27.280134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:27.280790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:27.281070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:27.281340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:27.282929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:27.354690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:27.391788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:27.393794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d3500-408d-470e-a497-b2999e6f2310 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:27.412523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 660/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:29.491617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec650-5cc2-4252-9627-e71e934689e1 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:29.492136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec650-5cc2-4252-9627-e71e934689e1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:29.492136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec650-5cc2-4252-9627-e71e934689e1 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:29.493450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 664/1324] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:29.498812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008330-732b-48e7-8e05-34692e3085a8 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:29.499265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008330-732b-48e7-8e05-34692e3085a8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:29.499403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b008330-732b-48e7-8e05-34692e3085a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:29.565457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-6b008330-732b-48e7-8e05-34692e3085a8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:29.566578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6b008330-732b-48e7-8e05-34692e3085a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:29.662250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 661/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:29.713238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0322e5-bf50-4032-8979-ad731a348e61 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:29.713516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0322e5-bf50-4032-8979-ad731a348e61 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:29.713785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0322e5-bf50-4032-8979-ad731a348e61 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:29.715120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 665/1326] 10.222.0.15 () {54 vars in 965 bytes} [Mon Aug 8 00:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:29.720917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310c578-ee2e-4d8e-ab69-83a956ffbb10 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:29.721744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310c578-ee2e-4d8e-ab69-83a956ffbb10 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:29.722134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310c578-ee2e-4d8e-ab69-83a956ffbb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:29.753245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:29.798064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4310c578-ee2e-4d8e-ab69-83a956ffbb10 None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:29.799828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4310c578-ee2e-4d8e-ab69-83a956ffbb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:29.875875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 662/1327] 10.222.0.15 () {58 vars in 1062 bytes} [Mon Aug 8 00:24:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:24:29.884759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0322e5-bf50-4032-8979-ad731a348e61 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:29.886297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8a0322e5-bf50-4032-8979-ad731a348e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:29.994604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0322e5-bf50-4032-8979-ad731a348e61 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:30.069083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:30.069606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:30.069882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:30.070150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:30.071774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:30.090400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:30.186663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:30.188142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb76c3-88ef-4d84-b046-f6e16c57bfcd service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:30.207194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 666/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:32.501451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036d365-2207-4906-a85a-55db732f962a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:32.502002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036d365-2207-4906-a85a-55db732f962a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:32.502002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036d365-2207-4906-a85a-55db732f962a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:32.504068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 663/1329] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:32.509400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe105f6-930c-474b-8a61-c922ddcff72a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:32.509842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe105f6-930c-474b-8a61-c922ddcff72a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:32.509961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe105f6-930c-474b-8a61-c922ddcff72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:32.578279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-afe105f6-930c-474b-8a61-c922ddcff72a None None] MFA Rules not processed for user `49a016ba549949ef81523de0e9a64871`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:32.579445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-afe105f6-930c-474b-8a61-c922ddcff72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:32.646644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:32.683024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 667/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:32.720420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036d365-2207-4906-a85a-55db732f962a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:32.722606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8036d365-2207-4906-a85a-55db732f962a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:32.851776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036d365-2207-4906-a85a-55db732f962a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:32.872924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:32.926602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:32.927201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:32.927508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:32.927776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:32.929364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:33.037304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:33.038654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548b540-2c63-4b3d-a4ce-c9d9f7e311b0 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:33.056321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 664/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:35.532867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929ad53-893d-4888-8a9d-0a9de438d201 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:35.533443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929ad53-893d-4888-8a9d-0a9de438d201 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:35.533443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929ad53-893d-4888-8a9d-0a9de438d201 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:35.535182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 668/1332] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:35.540683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f412f1-a9ce-4963-955f-d549cf24c89e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:35.541106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f412f1-a9ce-4963-955f-d549cf24c89e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:35.541221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f412f1-a9ce-4963-955f-d549cf24c89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:35.613162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:35.616375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-94f412f1-a9ce-4963-955f-d549cf24c89e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:35.617644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-94f412f1-a9ce-4963-955f-d549cf24c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:35.728629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 665/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:35.766728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929ad53-893d-4888-8a9d-0a9de438d201 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:35.768309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e929ad53-893d-4888-8a9d-0a9de438d201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:35.870785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929ad53-893d-4888-8a9d-0a9de438d201 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:35.945280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:35.945884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:35.946164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:35.946421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:35.948310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:35.975787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:36.062359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:36.063818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e2a77-3bc4-4b23-b334-a4351986844a service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:36.083719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 669/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:38.146688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5e348-740c-4fdc-86bd-92d5821647cb None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:38.147259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5e348-740c-4fdc-86bd-92d5821647cb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:38.147259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5e348-740c-4fdc-86bd-92d5821647cb None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:38.148619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 666/1335] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:38.153799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae2328-0b45-4585-9dde-9efc32937f2b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:38.154211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae2328-0b45-4585-9dde-9efc32937f2b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:38.154419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae2328-0b45-4585-9dde-9efc32937f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:38.218603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-96ae2328-0b45-4585-9dde-9efc32937f2b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:38.219835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-96ae2328-0b45-4585-9dde-9efc32937f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:38.292757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:38.315717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 670/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:38.356347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5e348-740c-4fdc-86bd-92d5821647cb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:38.357893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-82b5e348-740c-4fdc-86bd-92d5821647cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:38.460765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5e348-740c-4fdc-86bd-92d5821647cb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:38.488740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:38.532543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:38.533139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:38.533456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:38.533724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:38.535317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:38.641909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:38.643367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500dee5-32da-457f-b191-0169aec46123 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:38.662787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 667/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:41.547655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0357d-d7dc-47e2-a5d6-a1eaf847d1ed None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:41.548178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0357d-d7dc-47e2-a5d6-a1eaf847d1ed None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:41.548178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0357d-d7dc-47e2-a5d6-a1eaf847d1ed None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:41.549496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 671/1338] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:41.554850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289dc5b-e46e-4904-8c00-76d74e2b8886 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:41.555294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289dc5b-e46e-4904-8c00-76d74e2b8886 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:41.555446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289dc5b-e46e-4904-8c00-76d74e2b8886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:41.621452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1289dc5b-e46e-4904-8c00-76d74e2b8886 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:41.622635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1289dc5b-e46e-4904-8c00-76d74e2b8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:41.631290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:41.719685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 668/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:41.756754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0357d-d7dc-47e2-a5d6-a1eaf847d1ed None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:41.758303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-49e0357d-d7dc-47e2-a5d6-a1eaf847d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:41.857480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0357d-d7dc-47e2-a5d6-a1eaf847d1ed None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:41.927486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:41.928082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:41.928355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:41.928620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:41.930170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:41.964794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:42.036738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:42.038164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da28c6-2fe8-4810-9693-c14283a478e3 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:42.057734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 672/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:46.390924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab21795-5d0a-483b-9795-c1ca326e5ee2 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:46.391447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab21795-5d0a-483b-9795-c1ca326e5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:46.391545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab21795-5d0a-483b-9795-c1ca326e5ee2 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:46.393427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 669/1341] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:46.398795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba839db-da5f-476d-ae34-27d4ce578db0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:46.399263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba839db-da5f-476d-ae34-27d4ce578db0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:46.399503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba839db-da5f-476d-ae34-27d4ce578db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:46.468548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7ba839db-da5f-476d-ae34-27d4ce578db0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:46.469773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7ba839db-da5f-476d-ae34-27d4ce578db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:46.557098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:46.568505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 673/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:46.607834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab21795-5d0a-483b-9795-c1ca326e5ee2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:46.609363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4ab21795-5d0a-483b-9795-c1ca326e5ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:46.722667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab21795-5d0a-483b-9795-c1ca326e5ee2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:46.760879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:46.795884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:46.796489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:46.796785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:46.797051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:46.799146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:46.905012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:46.906357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6fa03-af39-405b-a605-fa258b8c7fc0 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:46.925889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 670/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:49.918802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bbeaf-31a4-4757-9743-1715d15b0f2a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:49.919413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bbeaf-31a4-4757-9743-1715d15b0f2a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:49.919413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bbeaf-31a4-4757-9743-1715d15b0f2a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:49.920773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 674/1344] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:49.925907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825099d-1b3c-4131-ab3a-e74e88337c27 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:49.926351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825099d-1b3c-4131-ab3a-e74e88337c27 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:49.926498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825099d-1b3c-4131-ab3a-e74e88337c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:49.991982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a825099d-1b3c-4131-ab3a-e74e88337c27 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:49.993103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a825099d-1b3c-4131-ab3a-e74e88337c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:50.011437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:50.090011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 671/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:50.128071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bbeaf-31a4-4757-9743-1715d15b0f2a None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:50.129628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-935bbeaf-31a4-4757-9743-1715d15b0f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:50.232173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bbeaf-31a4-4757-9743-1715d15b0f2a None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:50.305162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:50.305765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:50.306045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:50.306316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:50.307950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:50.351920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:50.420654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:50.422152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3b2e4-442d-40f5-a745-24f87a49d42f service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:50.443896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 675/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:24:54.849070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624dda-6037-4353-97e2-098397a0e1c0 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:54.849593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624dda-6037-4353-97e2-098397a0e1c0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:54.849593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad624dda-6037-4353-97e2-098397a0e1c0 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:54.850938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 672/1347] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:24:54.856387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62858de-14b8-48c8-a3db-1edc3cb26bb2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:54.856826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62858de-14b8-48c8-a3db-1edc3cb26bb2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:54.856943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62858de-14b8-48c8-a3db-1edc3cb26bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:54.928613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f62858de-14b8-48c8-a3db-1edc3cb26bb2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:24:54.929867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f62858de-14b8-48c8-a3db-1edc3cb26bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:55.033415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:55.036194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 676/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:24:55.074482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624dda-6037-4353-97e2-098397a0e1c0 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:24:55.076115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ad624dda-6037-4353-97e2-098397a0e1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:55.191954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad624dda-6037-4353-97e2-098397a0e1c0 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:24:55.246155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:24:55.282066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:24:55.282066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:24:55.282066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:24:55.282066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:24:55.282726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:24:55.402503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:24:55.404067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ef9ec-160e-41cf-a667-ca84716c6bc8 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:24:55.422910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 673/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:00.303322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f790fe-ccb2-47e1-806d-77ff9a32e1e5 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:00.304122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f790fe-ccb2-47e1-806d-77ff9a32e1e5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:00.304655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f790fe-ccb2-47e1-806d-77ff9a32e1e5 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:00.306491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 677/1350] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:25:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:00.322696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39316-0eb1-4f1e-8e6e-c76b1a1c9e46 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:00.323485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39316-0eb1-4f1e-8e6e-c76b1a1c9e46 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:00.323569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39316-0eb1-4f1e-8e6e-c76b1a1c9e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:00.461488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-93c39316-0eb1-4f1e-8e6e-c76b1a1c9e46 None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:00.462705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-93c39316-0eb1-4f1e-8e6e-c76b1a1c9e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:00.484304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:00.594204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 674/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:25:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:04.782150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebad6c-b4d9-4cd4-8af4-25f512df9d01 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:04.783196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebad6c-b4d9-4cd4-8af4-25f512df9d01 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:04.783685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebad6c-b4d9-4cd4-8af4-25f512df9d01 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:04.785485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 678/1352] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:04.790759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa1493-83b6-4c19-aab4-c0396c4676de None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:04.791191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa1493-83b6-4c19-aab4-c0396c4676de None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:04.791385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa1493-83b6-4c19-aab4-c0396c4676de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:04.870089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-daaa1493-83b6-4c19-aab4-c0396c4676de None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:04.871313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-daaa1493-83b6-4c19-aab4-c0396c4676de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:04.946142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:04.983163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 675/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:25:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:06.462759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c944a-94af-48bc-837f-c2ef7e5a9d8a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:06.463340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c944a-94af-48bc-837f-c2ef7e5a9d8a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:06.463340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c944a-94af-48bc-837f-c2ef7e5a9d8a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:06.464666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 679/1354] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:06.469579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ee50e-c8c2-471b-a4f9-71b08b1dae34 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:06.470552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ee50e-c8c2-471b-a4f9-71b08b1dae34 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:06.471013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ee50e-c8c2-471b-a4f9-71b08b1dae34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:06.547785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cf3ee50e-c8c2-471b-a4f9-71b08b1dae34 None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:06.549530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cf3ee50e-c8c2-471b-a4f9-71b08b1dae34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:06.662087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 676/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:25:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:10.088138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66f11b-7684-47ac-8bfb-17132e64e7bf None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:10.088667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66f11b-7684-47ac-8bfb-17132e64e7bf None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:10.088667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66f11b-7684-47ac-8bfb-17132e64e7bf None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:10.090005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 680/1356] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:10.096543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58936ed-7a31-473f-8a06-6e9c81878719 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:10.097354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58936ed-7a31-473f-8a06-6e9c81878719 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:10.097633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58936ed-7a31-473f-8a06-6e9c81878719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:10.154208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:10.207296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b58936ed-7a31-473f-8a06-6e9c81878719 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:10.210325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b58936ed-7a31-473f-8a06-6e9c81878719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:10.318573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 677/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:10.377696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd446aa1-7c67-4d7d-b558-57221163cde9 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:10.378276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd446aa1-7c67-4d7d-b558-57221163cde9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:10.378940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd446aa1-7c67-4d7d-b558-57221163cde9 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:10.380841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 681/1358] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 00:25:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:10.389475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aafe7-b744-4626-9d82-a840e1b4c461 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:10.391444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aafe7-b744-4626-9d82-a840e1b4c461 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:10.391444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aafe7-b744-4626-9d82-a840e1b4c461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:10.478054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-846aafe7-b744-4626-9d82-a840e1b4c461 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:10.479370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-846aafe7-b744-4626-9d82-a840e1b4c461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:10.540432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:10.619827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 678/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 00:25:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:10.629644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd446aa1-7c67-4d7d-b558-57221163cde9 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:10.631349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dd446aa1-7c67-4d7d-b558-57221163cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:10.788906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd446aa1-7c67-4d7d-b558-57221163cde9 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:10.867617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:10.911393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:10.911990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:10.912270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:10.912541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:10.914160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:11.014432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:11.016190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a2c2f1-46e9-430a-98c4-63314debd1ee service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:11.035591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 682/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:13.254453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac00069-69be-4348-aa87-8cc30b1f624f None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:13.255017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac00069-69be-4348-aa87-8cc30b1f624f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:13.255017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac00069-69be-4348-aa87-8cc30b1f624f None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:13.256545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 679/1361] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:13.261764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e62f-9016-4b25-926e-c5af81e767a1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:13.262566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e62f-9016-4b25-926e-c5af81e767a1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:13.262844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e62f-9016-4b25-926e-c5af81e767a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:13.333619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5326e62f-9016-4b25-926e-c5af81e767a1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:13.334855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5326e62f-9016-4b25-926e-c5af81e767a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:13.379353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:13.444202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 683/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:13.493640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac00069-69be-4348-aa87-8cc30b1f624f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:13.495155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2ac00069-69be-4348-aa87-8cc30b1f624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:13.659350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac00069-69be-4348-aa87-8cc30b1f624f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:13.671846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:13.927653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:13.928255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:13.928527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:13.928792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:13.930394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:14.023388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:14.024977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372d0b4-00d6-4d64-8e49-146e80e2d8d4 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:14.044409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 680/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:16.582166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77efe0-9e4a-4253-9b27-47f548e44a8b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:16.582705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77efe0-9e4a-4253-9b27-47f548e44a8b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:16.582705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77efe0-9e4a-4253-9b27-47f548e44a8b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:16.584069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 684/1364] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:16.589261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932f92c-3ff0-4cfe-bd0d-a13cba33ea94 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:16.589667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932f92c-3ff0-4cfe-bd0d-a13cba33ea94 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:16.589838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932f92c-3ff0-4cfe-bd0d-a13cba33ea94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:16.605671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:16.659773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3932f92c-3ff0-4cfe-bd0d-a13cba33ea94 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:16.660979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3932f92c-3ff0-4cfe-bd0d-a13cba33ea94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:16.764644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 681/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:16.810712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e676bdf-051c-42af-9291-6f9a8274dcaa None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:16.811177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e676bdf-051c-42af-9291-6f9a8274dcaa None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:16.811452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e676bdf-051c-42af-9291-6f9a8274dcaa None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:16.814887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 685/1366] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 00:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:16.821201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a62b-046a-43a6-b440-83a0b8bdb2bf None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:16.822271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a62b-046a-43a6-b440-83a0b8bdb2bf None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:16.822271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a62b-046a-43a6-b440-83a0b8bdb2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:16.904761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9046a62b-046a-43a6-b440-83a0b8bdb2bf None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:16.905959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9046a62b-046a-43a6-b440-83a0b8bdb2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:16.948919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:17.034702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 682/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 00:25:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:17.042553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e676bdf-051c-42af-9291-6f9a8274dcaa None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:17.044094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1e676bdf-051c-42af-9291-6f9a8274dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:17.195473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e676bdf-051c-42af-9291-6f9a8274dcaa None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:17.216936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:17.314084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:17.314905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:17.315184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:17.315480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:17.317109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:17.415118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:17.416749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128713-80a3-422a-abbf-a33016cf79d8 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:17.435285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 686/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:18.288409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48a6f-7622-496a-a481-5a5912ea2f09 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:18.288993 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48a6f-7622-496a-a481-5a5912ea2f09 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:18.288993 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb48a6f-7622-496a-a481-5a5912ea2f09 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:18.290494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 683/1369] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:18.295568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c61dc7-6133-4bbb-bf2f-61e85a27d2c6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:18.295989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c61dc7-6133-4bbb-bf2f-61e85a27d2c6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:18.296165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c61dc7-6133-4bbb-bf2f-61e85a27d2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:18.330435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:18.368865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-98c61dc7-6133-4bbb-bf2f-61e85a27d2c6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:18.370857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-98c61dc7-6133-4bbb-bf2f-61e85a27d2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:18.486107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 687/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:18.832150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706340a-215d-4164-bce0-02426ae2ae1a None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:18.832150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706340a-215d-4164-bce0-02426ae2ae1a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:18.832150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706340a-215d-4164-bce0-02426ae2ae1a None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:18.832150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 684/1371] 10.222.0.15 () {54 vars in 964 bytes} [Mon Aug 8 00:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:18.836170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efb46c-62df-444f-857f-3f06979e030d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:18.836461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efb46c-62df-444f-857f-3f06979e030d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:18.836737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efb46c-62df-444f-857f-3f06979e030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:18.915042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-20efb46c-62df-444f-857f-3f06979e030d None None] MFA Rules not processed for user `2e66698136f4429b8df67e5050a8da2f`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:18.916299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-20efb46c-62df-444f-857f-3f06979e030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:18.965977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:18.998938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 688/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Aug 8 00:25:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:19.006690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706340a-215d-4164-bce0-02426ae2ae1a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:19.008220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9706340a-215d-4164-bce0-02426ae2ae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:19.117205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706340a-215d-4164-bce0-02426ae2ae1a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:19.159026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:19.198861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:19.199546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:19.199832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:19.200026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:19.201633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:19.325482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:19.326581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfe14f5-3e76-47c4-8e01-939291257362 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:19.348455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 685/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:19.670295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981b06-2082-4b7e-ae83-551a2603e317 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:19.670934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981b06-2082-4b7e-ae83-551a2603e317 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:19.670934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981b06-2082-4b7e-ae83-551a2603e317 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:19.672502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 689/1374] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:19.678243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47c49-4d27-4be8-ae25-3e861664e99e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:19.678931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47c49-4d27-4be8-ae25-3e861664e99e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:19.679353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47c49-4d27-4be8-ae25-3e861664e99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:19.746125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1fb47c49-4d27-4be8-ae25-3e861664e99e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:19.747252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1fb47c49-4d27-4be8-ae25-3e861664e99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:19.768499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:19.844786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 686/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:19.889242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981b06-2082-4b7e-ae83-551a2603e317 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:19.890757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8b981b06-2082-4b7e-ae83-551a2603e317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:20.038687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981b06-2082-4b7e-ae83-551a2603e317 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:20.089401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:20.153298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:20.153904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:20.154181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:20.154452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:20.156160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:20.248867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:20.250504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bbc94-872d-4359-b61c-9bf71e9ef5d3 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:20.268610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 690/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:22.509179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f69c-f211-4cd4-a6b2-f3ef64716270 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:22.509745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f69c-f211-4cd4-a6b2-f3ef64716270 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:22.509745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f69c-f211-4cd4-a6b2-f3ef64716270 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:22.511051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 687/1377] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:22.519267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983de66-1260-45a7-b891-5f74e37c23a8 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:22.519267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983de66-1260-45a7-b891-5f74e37c23a8 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:22.519267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983de66-1260-45a7-b891-5f74e37c23a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:22.602493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5983de66-1260-45a7-b891-5f74e37c23a8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:22.603721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5983de66-1260-45a7-b891-5f74e37c23a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:22.611533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:22.704122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 691/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:22.747436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f69c-f211-4cd4-a6b2-f3ef64716270 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:22.748935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0011f69c-f211-4cd4-a6b2-f3ef64716270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:22.894696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f69c-f211-4cd4-a6b2-f3ef64716270 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:22.906301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:23.049775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:23.050946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:23.051435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:23.051876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:23.053952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:23.155361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:23.157560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4017cb3-bc06-490c-8283-e22c823c2b2e service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:23.177573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 688/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:25.387219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6ae3-949c-4e4b-95b5-9278cdcb36b0 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:25.387871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6ae3-949c-4e4b-95b5-9278cdcb36b0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:25.387871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b6ae3-949c-4e4b-95b5-9278cdcb36b0 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:25.389276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 692/1380] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:25.394486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667b2ee-be48-452c-94d9-a54d36d250b9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:25.394938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667b2ee-be48-452c-94d9-a54d36d250b9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:25.395041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667b2ee-be48-452c-94d9-a54d36d250b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:25.406351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:25.464338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1667b2ee-be48-452c-94d9-a54d36d250b9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:25.465438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1667b2ee-be48-452c-94d9-a54d36d250b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:25.568052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 689/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:25.613341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b6ae3-949c-4e4b-95b5-9278cdcb36b0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:25.614847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-994b6ae3-949c-4e4b-95b5-9278cdcb36b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:25.770105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:25.782933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b6ae3-949c-4e4b-95b5-9278cdcb36b0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:25.918688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:25.919327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:25.919608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:25.919886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:25.921503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:26.017123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:26.018710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51123fa-5452-45c1-9170-16fc6bec8f53 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:26.029722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:26.038593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 693/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:28.353929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc06a2f-f180-4401-8b9d-60727f1bb7bc None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:28.354519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc06a2f-f180-4401-8b9d-60727f1bb7bc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:28.354691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc06a2f-f180-4401-8b9d-60727f1bb7bc None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:28.356599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 690/1383] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:28.361948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607925-1976-4241-bf67-ab0e47c9f2d2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:28.362475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607925-1976-4241-bf67-ab0e47c9f2d2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:28.362475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce607925-1976-4241-bf67-ab0e47c9f2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:28.428893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ce607925-1976-4241-bf67-ab0e47c9f2d2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:28.430159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ce607925-1976-4241-bf67-ab0e47c9f2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:28.530104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 694/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:28.577581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc06a2f-f180-4401-8b9d-60727f1bb7bc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:28.579184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fbc06a2f-f180-4401-8b9d-60727f1bb7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:28.663479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:28.729976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc06a2f-f180-4401-8b9d-60727f1bb7bc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:28.855361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:28.856007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:28.856335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:28.856626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:28.858225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:28.911507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:28.954139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:28.955605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afdbd4-49ee-41b1-87ad-3136ec89b298 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:28.974029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 691/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:31.053339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a163e-d12b-4c7c-8d8b-21b1564719fa None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:31.053898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a163e-d12b-4c7c-8d8b-21b1564719fa None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:31.053898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a163e-d12b-4c7c-8d8b-21b1564719fa None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:31.055249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 695/1386] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:31.060802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682801-5081-43ea-a419-d732fb772e43 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:31.061401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682801-5081-43ea-a419-d732fb772e43 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:31.061401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682801-5081-43ea-a419-d732fb772e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:31.129592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-54682801-5081-43ea-a419-d732fb772e43 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:31.130859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-54682801-5081-43ea-a419-d732fb772e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:31.235819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 692/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:31.279751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a163e-d12b-4c7c-8d8b-21b1564719fa None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:31.281379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-737a163e-d12b-4c7c-8d8b-21b1564719fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:31.365118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:31.454584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a163e-d12b-4c7c-8d8b-21b1564719fa None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:31.577337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:31.577784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:31.578062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:31.578336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:31.580207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:31.622339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:31.685038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:31.687123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbec97-27ce-4453-98fe-ceb58686eadb service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:31.711586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 696/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:33.969236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab1d94-b97f-418d-9e92-99ee60b78bfd None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:33.969771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab1d94-b97f-418d-9e92-99ee60b78bfd None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:33.970076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab1d94-b97f-418d-9e92-99ee60b78bfd None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:33.971560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 693/1389] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:33.976874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae148ed-dda6-424c-9017-8aa274032e42 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:33.977283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae148ed-dda6-424c-9017-8aa274032e42 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:33.977435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae148ed-dda6-424c-9017-8aa274032e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:34.044511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-bae148ed-dda6-424c-9017-8aa274032e42 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:34.045739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bae148ed-dda6-424c-9017-8aa274032e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:34.144702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 697/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:34.190987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab1d94-b97f-418d-9e92-99ee60b78bfd None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:34.192556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-98ab1d94-b97f-418d-9e92-99ee60b78bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:34.226427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:34.342996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab1d94-b97f-418d-9e92-99ee60b78bfd None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:34.444437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:34.456059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:34.456668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:34.456944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:34.457212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:34.458835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:34.550927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:34.552539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c4209c-fe2c-4cc4-b3cf-793840e5ae40 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:34.570574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 694/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:34.724591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d12efa-f491-4cf8-affb-44146e4bfce4 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:34.724878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d12efa-f491-4cf8-affb-44146e4bfce4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:34.725151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d12efa-f491-4cf8-affb-44146e4bfce4 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:34.726563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 698/1392] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:34.731998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da553f4-81a3-4a8c-95c6-d483200e6607 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:34.732282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da553f4-81a3-4a8c-95c6-d483200e6607 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:34.732555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da553f4-81a3-4a8c-95c6-d483200e6607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:34.808793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7da553f4-81a3-4a8c-95c6-d483200e6607 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:34.809986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7da553f4-81a3-4a8c-95c6-d483200e6607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:34.874150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:34.914300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 695/1393] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:25:34.971237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8907e-d4e9-4cde-afe7-d532cc4fc256 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:34.971537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8907e-d4e9-4cde-afe7-d532cc4fc256 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:34.971811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8907e-d4e9-4cde-afe7-d532cc4fc256 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:34.973133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 699/1394] 10.222.0.15 () {54 vars in 966 bytes} [Mon Aug 8 00:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:34.978063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc4b8d-85ed-4c76-a591-9b2d346d68d0 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:34.978498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc4b8d-85ed-4c76-a591-9b2d346d68d0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:34.978635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc4b8d-85ed-4c76-a591-9b2d346d68d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:35.055987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-56dc4b8d-85ed-4c76-a591-9b2d346d68d0 None None] MFA Rules not processed for user `de6c52a7640e47c7be2253e32aa5a173`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:35.057206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-56dc4b8d-85ed-4c76-a591-9b2d346d68d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:35.137072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 696/1395] 10.222.0.15 () {58 vars in 1063 bytes} [Mon Aug 8 00:25:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:35.145439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8907e-d4e9-4cde-afe7-d532cc4fc256 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:35.146954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b7b8907e-d4e9-4cde-afe7-d532cc4fc256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:35.160828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:35.253448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8907e-d4e9-4cde-afe7-d532cc4fc256 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:35.330451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:35.331056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:35.331365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:35.331639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:35.333221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:35.402848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:35.444433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:35.445896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4c0ed-c003-4a76-9a11-703f19bc3c5a service glance] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:35.463844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 700/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:36.489280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2d27a-3427-4adf-8e53-fbfede66f026 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:36.490159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2d27a-3427-4adf-8e53-fbfede66f026 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:36.490159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2d27a-3427-4adf-8e53-fbfede66f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:36.562469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-97e2d27a-3427-4adf-8e53-fbfede66f026 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:36.564400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-97e2d27a-3427-4adf-8e53-fbfede66f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:36.573293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:36.667075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 697/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:25:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:25:36.713141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130a90a-9681-446d-aec6-ebc20f36275c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:36.713668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130a90a-9681-446d-aec6-ebc20f36275c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:36.713668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130a90a-9681-446d-aec6-ebc20f36275c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:36.715853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 701/1398] 10.222.0.15 () {54 vars in 943 bytes} [Mon Aug 8 00:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:36.724076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:36.724440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:36.724722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:36.775765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1c3a-b06f-47fc-849d-554d969c502a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:36.776029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1c3a-b06f-47fc-849d-554d969c502a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:36.776303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1c3a-b06f-47fc-849d-554d969c502a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:36.777641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 702/1399] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:25:36.783346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16931b61-1460-413a-a913-e660a54013ec None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:36.783658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16931b61-1460-413a-a913-e660a54013ec None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:36.783908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16931b61-1460-413a-a913-e660a54013ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:36.813393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] MFA Rules not processed for user `aaf8d31c016449e6aa6408c7f69abe20`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:36.814674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:36.853268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-16931b61-1460-413a-a913-e660a54013ec None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:36.854500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-16931b61-1460-413a-a913-e660a54013ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:36.915913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:36.940897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 698/1400] 10.222.0.15 () {58 vars in 1040 bytes} [Mon Aug 8 00:25:36 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:36.950684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:36.962116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed082142-4ac8-45e8-90c4-3ca809d4018c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:36.965536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 703/1401] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:25:37.017310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16931b61-1460-413a-a913-e660a54013ec None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:37.019145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-16931b61-1460-413a-a913-e660a54013ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:37.051802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:37.079466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:37.079989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:37.080329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:37.080598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:37.082269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:37.149243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16931b61-1460-413a-a913-e660a54013ec None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:37.188459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:37.190071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d35cf95-1038-4b82-a37d-aae883e2840b service swift] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:37.199143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 699/1402] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:25:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 00:25:37.280105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:37.280711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:37.280992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:37.281263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:37.283838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:37.302727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:37.374857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:37.376754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12204aa8-4ddd-46f4-9fef-9d09dbfd9bcf service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:37.394762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 704/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:25:39.623552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceca9b3-4567-41b4-bc7f-d159c51446fa None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:39.624088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceca9b3-4567-41b4-bc7f-d159c51446fa None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:39.624088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceca9b3-4567-41b4-bc7f-d159c51446fa None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:39.625980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 700/1404] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:39.631445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40237c64-f0a7-4e3d-b760-6efbc16b25d5 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:39.631867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40237c64-f0a7-4e3d-b760-6efbc16b25d5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:39.632042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40237c64-f0a7-4e3d-b760-6efbc16b25d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:39.708670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-40237c64-f0a7-4e3d-b760-6efbc16b25d5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:39.709963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-40237c64-f0a7-4e3d-b760-6efbc16b25d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:39.817942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:39.831285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 705/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:39.871168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceca9b3-4567-41b4-bc7f-d159c51446fa None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:39.872761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3ceca9b3-4567-41b4-bc7f-d159c51446fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:39.886367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:40.035387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceca9b3-4567-41b4-bc7f-d159c51446fa None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:40.123948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:40.161005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:40.161531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:40.161807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:40.162073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:40.163729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:40.272502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:40.274263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b48e1-a6b3-45dc-9698-b2a01ae5e283 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:40.313055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 701/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:43.320483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a19940-009f-439a-b342-2754ea9f05da None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:43.328285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a19940-009f-439a-b342-2754ea9f05da None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:43.328285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a19940-009f-439a-b342-2754ea9f05da None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:43.328285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 706/1407] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:25:43.328737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692c260-e3ec-4912-b70b-78ea90806662 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:43.329159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692c260-e3ec-4912-b70b-78ea90806662 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:43.329315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692c260-e3ec-4912-b70b-78ea90806662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:43.410961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2692c260-e3ec-4912-b70b-78ea90806662 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:25:43.412215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2692c260-e3ec-4912-b70b-78ea90806662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:43.465994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:43.521658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 702/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:25:43.566474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a19940-009f-439a-b342-2754ea9f05da None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:25:43.568039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d9a19940-009f-439a-b342-2754ea9f05da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:43.736196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a19940-009f-439a-b342-2754ea9f05da None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:25:43.813463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:43.865163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:25:43.865992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:25:43.866375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:25:43.866709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:25:43.868412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:25:43.965147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:25:43.966843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ed5b5-2636-4ff9-a14b-7a821140c8bf service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:25:43.986968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 707/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:25:53.521836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=23 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:25:53.983577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=64 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:00.379891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e58cb-1317-4408-bc1b-a627d726aae8 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:00.380486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e58cb-1317-4408-bc1b-a627d726aae8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:00.380486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e58cb-1317-4408-bc1b-a627d726aae8 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:00.381652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 703/1410] 10.222.0.15 () {54 vars in 918 bytes} [Mon Aug 8 00:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:00.386659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15d57-de16-4249-92a0-8ccf420ea031 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:00.387099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15d57-de16-4249-92a0-8ccf420ea031 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:00.387253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d15d57-de16-4249-92a0-8ccf420ea031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:00.461667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-31d15d57-de16-4249-92a0-8ccf420ea031 None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:00.462853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-31d15d57-de16-4249-92a0-8ccf420ea031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:00.568868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 708/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 8 00:26:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:00.584091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f53b1-e8a9-48ff-98e3-728b21edd9ea None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:00.584370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f53b1-e8a9-48ff-98e3-728b21edd9ea None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:00.588519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f53b1-e8a9-48ff-98e3-728b21edd9ea None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:00.588519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 704/1412] 10.222.0.15 () {54 vars in 918 bytes} [Mon Aug 8 00:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:00.590520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4082-f37b-4fed-b1a4-e9f9c7942b0e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:00.590810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4082-f37b-4fed-b1a4-e9f9c7942b0e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:00.591086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4082-f37b-4fed-b1a4-e9f9c7942b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:00.665444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:00.668587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-182e4082-f37b-4fed-b1a4-e9f9c7942b0e None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:00.669809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-182e4082-f37b-4fed-b1a4-e9f9c7942b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:00.787285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 709/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Mon Aug 8 00:26:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:26:03.264518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf294da2-9ffd-4921-b8b2-8cf9f689bfd3 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:03.265059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf294da2-9ffd-4921-b8b2-8cf9f689bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:03.265059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf294da2-9ffd-4921-b8b2-8cf9f689bfd3 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:03.266368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:03.271665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd19107-3f0c-4e0a-a457-585d3cf1c695 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:03.272050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd19107-3f0c-4e0a-a457-585d3cf1c695 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:03.272260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd19107-3f0c-4e0a-a457-585d3cf1c695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:03.336623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1cd19107-3f0c-4e0a-a457-585d3cf1c695 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:03.337819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1cd19107-3f0c-4e0a-a457-585d3cf1c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:03.391593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:03.432525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:03.475154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf294da2-9ffd-4921-b8b2-8cf9f689bfd3 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:03.477044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bf294da2-9ffd-4921-b8b2-8cf9f689bfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:03.626395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf294da2-9ffd-4921-b8b2-8cf9f689bfd3 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:03.702091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:03.743359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:03.743898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:03.744177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:03.744455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:03.746053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:03.835214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:03.836847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5a4fa1-7af7-458b-98d9-79cf96c915ba service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:03.855195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 706/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:04.758282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481c7c5-1ac7-4967-a79d-873e808a9fad None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:04.758879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481c7c5-1ac7-4967-a79d-873e808a9fad None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:04.758879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481c7c5-1ac7-4967-a79d-873e808a9fad None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:04.760201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:04.765158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b3991-7667-4273-a5ac-dfb503ce5b81 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:04.765598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b3991-7667-4273-a5ac-dfb503ce5b81 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:04.765706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b3991-7667-4273-a5ac-dfb503ce5b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:04.842015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-522b3991-7667-4273-a5ac-dfb503ce5b81 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:04.843250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-522b3991-7667-4273-a5ac-dfb503ce5b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:04.870919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:04.940144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:05.093123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481c7c5-1ac7-4967-a79d-873e808a9fad None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:05.094651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2481c7c5-1ac7-4967-a79d-873e808a9fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:05.239816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481c7c5-1ac7-4967-a79d-873e808a9fad None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:05.279598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:05.350511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:05.351161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:05.351470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:05.351740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:05.353330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:05.440949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:05.442532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9ba2-00ef-48a5-921a-c4c6ab8b02bc service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:05.460565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 712/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:06.342116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af371ad8-e04d-46a0-824f-257f10a39bb0 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:06.342652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af371ad8-e04d-46a0-824f-257f10a39bb0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:06.342652 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af371ad8-e04d-46a0-824f-257f10a39bb0 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:06.344127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:06.349798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecad0f-0790-40fd-b46d-5fc0349995c0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:06.350249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecad0f-0790-40fd-b46d-5fc0349995c0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:06.350357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecad0f-0790-40fd-b46d-5fc0349995c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:06.404331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:06.426164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ebecad0f-0790-40fd-b46d-5fc0349995c0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:06.427420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ebecad0f-0790-40fd-b46d-5fc0349995c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:06.523978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:06.677514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af371ad8-e04d-46a0-824f-257f10a39bb0 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:06.679070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-af371ad8-e04d-46a0-824f-257f10a39bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:06.823422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af371ad8-e04d-46a0-824f-257f10a39bb0 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:06.835907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:06.936455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:06.937062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:06.937338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:06.937607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:06.939456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:07.027335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:07.028912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c88c69-7a13-4004-a637-fab6531521ab service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:07.047203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 709/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:07.907719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7546fe-20b6-4348-bbeb-179f6eedb17b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:07.908295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7546fe-20b6-4348-bbeb-179f6eedb17b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:07.908295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7546fe-20b6-4348-bbeb-179f6eedb17b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:07.909619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:07.914820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c633f-61cc-4bc6-ae36-8c9c741b6345 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:07.915219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c633f-61cc-4bc6-ae36-8c9c741b6345 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:07.915388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c633f-61cc-4bc6-ae36-8c9c741b6345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:07.934691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:07.990695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a72c633f-61cc-4bc6-ae36-8c9c741b6345 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:07.991908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a72c633f-61cc-4bc6-ae36-8c9c741b6345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:08.086527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:08.237969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7546fe-20b6-4348-bbeb-179f6eedb17b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:08.239449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1b7546fe-20b6-4348-bbeb-179f6eedb17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:08.350707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:08.386448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7546fe-20b6-4348-bbeb-179f6eedb17b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:08.618274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:08.618877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:08.619154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:08.619454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:08.621040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:08.696966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:08.710584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:08.712214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892d6491-6b28-451d-bf6b-d2711774f7ac service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:08.729942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 715/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:09.585076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf223-4171-4f3d-a454-d3cae2bfe5d3 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:09.585587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf223-4171-4f3d-a454-d3cae2bfe5d3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:09.585587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777cf223-4171-4f3d-a454-d3cae2bfe5d3 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:09.586868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:09.591772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc60860-e5df-4d3d-84fc-175efa57c26a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:09.592161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc60860-e5df-4d3d-84fc-175efa57c26a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:09.592308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc60860-e5df-4d3d-84fc-175efa57c26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:09.665963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ddc60860-e5df-4d3d-84fc-175efa57c26a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:09.667176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ddc60860-e5df-4d3d-84fc-175efa57c26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:09.763251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:09.915692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf223-4171-4f3d-a454-d3cae2bfe5d3 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:09.917233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-777cf223-4171-4f3d-a454-d3cae2bfe5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:09.999619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:10.060086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777cf223-4171-4f3d-a454-d3cae2bfe5d3 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:10.169341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:10.169858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:10.170129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:10.170393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:10.173233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:10.221118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:10.262403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:10.264032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131f781-3cbe-4185-9d4f-2a9789b3d26c service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:10.282566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 712/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:11.141694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ea80-a475-4b6f-8d4c-3c244a7a3b13 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:11.142253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ea80-a475-4b6f-8d4c-3c244a7a3b13 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:11.142253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ea80-a475-4b6f-8d4c-3c244a7a3b13 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:11.143626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:11.148470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db371f-f359-4d09-b84c-53ea80834292 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:11.148895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db371f-f359-4d09-b84c-53ea80834292 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:11.149014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db371f-f359-4d09-b84c-53ea80834292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:11.224621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b4db371f-f359-4d09-b84c-53ea80834292 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:11.225840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b4db371f-f359-4d09-b84c-53ea80834292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:11.319653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:11.469233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ea80-a475-4b6f-8d4c-3c244a7a3b13 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:11.471347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e914ea80-a475-4b6f-8d4c-3c244a7a3b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:11.519180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:11.616831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ea80-a475-4b6f-8d4c-3c244a7a3b13 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:11.730829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:11.731397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:11.731677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:11.731950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:11.733542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:11.740812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:11.821937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:11.823770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b201e-2297-4039-abd6-c7ae80701ba9 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:11.842728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 718/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:12.722557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcdf3e-3a92-4b59-8076-dacf3e3cda59 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:12.723098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcdf3e-3a92-4b59-8076-dacf3e3cda59 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:12.723098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fcdf3e-3a92-4b59-8076-dacf3e3cda59 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:12.724534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:26:12.729359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711edc9b-77f2-4f1c-9766-3cfd6771614d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:12.729754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711edc9b-77f2-4f1c-9766-3cfd6771614d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:12.729910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711edc9b-77f2-4f1c-9766-3cfd6771614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:12.804472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-711edc9b-77f2-4f1c-9766-3cfd6771614d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:12.805594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-711edc9b-77f2-4f1c-9766-3cfd6771614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:12.873391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:12.901596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:13.052963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcdf3e-3a92-4b59-8076-dacf3e3cda59 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:13.054542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d5fcdf3e-3a92-4b59-8076-dacf3e3cda59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:13.072150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:13.201127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fcdf3e-3a92-4b59-8076-dacf3e3cda59 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:13.287660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:13.312419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:13.312967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:13.313813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:13.314090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:13.315814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:13.406459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:13.408331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7e652-8eac-44b2-b151-1f68f0d1cf45 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:13.426449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 715/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:14.304290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee6ba4-31ea-4ebf-b864-8eb62e575711 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:14.304842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee6ba4-31ea-4ebf-b864-8eb62e575711 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:14.304842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee6ba4-31ea-4ebf-b864-8eb62e575711 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:14.306181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:14.311017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f04a0-f6f1-4879-8dc7-a5973c966c9a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:14.311460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f04a0-f6f1-4879-8dc7-a5973c966c9a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:14.311750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f04a0-f6f1-4879-8dc7-a5973c966c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:14.386166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-889f04a0-f6f1-4879-8dc7-a5973c966c9a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:14.387456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-889f04a0-f6f1-4879-8dc7-a5973c966c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:14.429196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:14.484834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:14.636141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee6ba4-31ea-4ebf-b864-8eb62e575711 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:14.641931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-52ee6ba4-31ea-4ebf-b864-8eb62e575711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:14.782872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee6ba4-31ea-4ebf-b864-8eb62e575711 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:14.835299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:14.893935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:14.894531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:14.894806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:14.895070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:14.896677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:14.982904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:14.984533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088ba8-5eb5-4e8d-8b51-93ef60a71b59 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:15.003010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 721/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:15.872513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a140-e8cc-48ac-8737-93821c9cb364 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:15.873069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a140-e8cc-48ac-8737-93821c9cb364 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:15.873069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53a140-e8cc-48ac-8737-93821c9cb364 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:15.874445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:15.879806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e120-bd09-4672-ab8e-93f0cb8e6878 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:15.880235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e120-bd09-4672-ab8e-93f0cb8e6878 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:15.880357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70e120-bd09-4672-ab8e-93f0cb8e6878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:15.954955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0b70e120-bd09-4672-ab8e-93f0cb8e6878 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:15.956089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0b70e120-bd09-4672-ab8e-93f0cb8e6878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:15.958073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:16.051907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:16.206485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53a140-e8cc-48ac-8737-93821c9cb364 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:16.207972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fe53a140-e8cc-48ac-8737-93821c9cb364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:16.354430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53a140-e8cc-48ac-8737-93821c9cb364 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:16.381393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:16.467035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:16.467575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:16.467849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:16.468132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:16.469742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:16.556963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:16.558544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600377b-0189-4bde-8bf8-b8eddf371a56 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:16.576739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 718/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:18.527537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c08ba-322d-4d76-a5db-23aa794b1b26 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:18.528195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c08ba-322d-4d76-a5db-23aa794b1b26 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:18.528195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c08ba-322d-4d76-a5db-23aa794b1b26 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:18.529422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:18.534932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b8844-5edc-4464-991d-939e8a6fed51 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:18.535410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b8844-5edc-4464-991d-939e8a6fed51 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:18.535583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b8844-5edc-4464-991d-939e8a6fed51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:18.575691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:18.600069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-585b8844-5edc-4464-991d-939e8a6fed51 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:18.601316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-585b8844-5edc-4464-991d-939e8a6fed51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:18.702324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:18.741608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c08ba-322d-4d76-a5db-23aa794b1b26 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:18.743168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-149c08ba-322d-4d76-a5db-23aa794b1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:18.846039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c08ba-322d-4d76-a5db-23aa794b1b26 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:18.878358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:18.918504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:18.919100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:18.919475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:18.919743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:18.921334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:19.027431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:19.028849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f972d-2903-4e42-b5af-9ddaf157bcf5 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:19.046600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 724/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:26:21.257175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483451a6-fb78-4a87-b26d-5bd390385580 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:21.257680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483451a6-fb78-4a87-b26d-5bd390385580 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:21.257785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483451a6-fb78-4a87-b26d-5bd390385580 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:21.259067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:21.264269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161362-df59-473e-bc3a-5dbccfa4120e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:21.264711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161362-df59-473e-bc3a-5dbccfa4120e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:21.264822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32161362-df59-473e-bc3a-5dbccfa4120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:21.332100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-32161362-df59-473e-bc3a-5dbccfa4120e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:21.333306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-32161362-df59-473e-bc3a-5dbccfa4120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:21.346338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:21.428394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:21.468363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483451a6-fb78-4a87-b26d-5bd390385580 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:21.470162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-483451a6-fb78-4a87-b26d-5bd390385580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:21.572271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483451a6-fb78-4a87-b26d-5bd390385580 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:21.585001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:21.645471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:21.646070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:21.646348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:21.646620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:21.648248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:21.757265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:21.758628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed146753-5100-4c16-a35b-5f1bae44574a service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:21.776430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 721/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:24.150065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424717d-aa7a-4837-8ca4-752b6065142a None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:24.150602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424717d-aa7a-4837-8ca4-752b6065142a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:24.150734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4424717d-aa7a-4837-8ca4-752b6065142a None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:24.152026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:24.157510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8117b53-00de-4c93-a9cf-f4ea65854679 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:24.157920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8117b53-00de-4c93-a9cf-f4ea65854679 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:24.158089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8117b53-00de-4c93-a9cf-f4ea65854679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:24.207400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:24.225019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d8117b53-00de-4c93-a9cf-f4ea65854679 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:24.226250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d8117b53-00de-4c93-a9cf-f4ea65854679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:24.322435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:24.362304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424717d-aa7a-4837-8ca4-752b6065142a None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:24.363876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4424717d-aa7a-4837-8ca4-752b6065142a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:24.465152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4424717d-aa7a-4837-8ca4-752b6065142a None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:24.503983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:24.537205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:24.537797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:24.538069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:24.538334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:24.539948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:24.645803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:24.647372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c520bcdb-9e2b-46c3-a47f-bc75676a60c8 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:24.666486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 727/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:26.798378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fe296-2884-4ca5-80ba-94f00cc62b7b None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:26.798955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fe296-2884-4ca5-80ba-94f00cc62b7b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:26.798955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0fe296-2884-4ca5-80ba-94f00cc62b7b None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:26.800308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:26.805629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ec865-0f8e-4d85-a046-a4b77cc19734 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:26.806074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ec865-0f8e-4d85-a046-a4b77cc19734 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:26.806216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ec865-0f8e-4d85-a046-a4b77cc19734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:26.871133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-9f3ec865-0f8e-4d85-a046-a4b77cc19734 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:26.872344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9f3ec865-0f8e-4d85-a046-a4b77cc19734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:26.893660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:26.968770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:27.008215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fe296-2884-4ca5-80ba-94f00cc62b7b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:27.009738 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-db0fe296-2884-4ca5-80ba-94f00cc62b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:27.113663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0fe296-2884-4ca5-80ba-94f00cc62b7b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:27.133972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:27.188369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:27.189115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:27.189420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:27.189712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:27.191320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:27.299877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:27.301237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd587bc3-d2d6-49ef-999e-b295cad9a88c service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:27.320061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 724/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:29.465229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf939-49b1-49fb-a9ad-81f704da45d4 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:29.465799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf939-49b1-49fb-a9ad-81f704da45d4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:29.465799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf939-49b1-49fb-a9ad-81f704da45d4 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:29.467020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:29.472419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525a4ab-946c-4fe2-80b6-700c3638da20 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:29.472827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525a4ab-946c-4fe2-80b6-700c3638da20 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:29.472961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525a4ab-946c-4fe2-80b6-700c3638da20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:29.535387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:29.537876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-5525a4ab-946c-4fe2-80b6-700c3638da20 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:29.539078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5525a4ab-946c-4fe2-80b6-700c3638da20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:29.634246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:29.674374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf939-49b1-49fb-a9ad-81f704da45d4 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:29.675979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-64ebf939-49b1-49fb-a9ad-81f704da45d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:29.777362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf939-49b1-49fb-a9ad-81f704da45d4 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:29.827177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:29.851319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:29.851925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:29.852218 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:29.852471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:29.854112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:29.961633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:29.963044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edbecd-8961-42b7-bd13-d085e539f1ac service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:29.980958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 730/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:32.018963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66867148-76d2-4412-8186-5e24876749ce None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:32.019524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66867148-76d2-4412-8186-5e24876749ce None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:32.019524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66867148-76d2-4412-8186-5e24876749ce None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:32.020796 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:32.026082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68729a96-fb81-4dad-9678-ebf0039e06a4 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:32.026469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68729a96-fb81-4dad-9678-ebf0039e06a4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:32.026636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68729a96-fb81-4dad-9678-ebf0039e06a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:32.094448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-68729a96-fb81-4dad-9678-ebf0039e06a4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:32.095957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-68729a96-fb81-4dad-9678-ebf0039e06a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:32.125785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:32.193234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:32.232581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66867148-76d2-4412-8186-5e24876749ce None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:32.234117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-66867148-76d2-4412-8186-5e24876749ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:32.337344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66867148-76d2-4412-8186-5e24876749ce None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:32.366308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:32.411634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:32.412242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:32.412517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:32.412786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:32.414383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:32.520525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:32.521950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e4503-53b7-46a2-9bb9-b34727776fc3 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:32.540696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 727/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:34.606254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2274930-b3d7-4a84-a7b6-01b6f63c1e5b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:34.606821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2274930-b3d7-4a84-a7b6-01b6f63c1e5b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:34.606821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2274930-b3d7-4a84-a7b6-01b6f63c1e5b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:34.608193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:34.613870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1d932-4bb8-4927-a0d9-9e307e1808f4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:34.614295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1d932-4bb8-4927-a0d9-9e307e1808f4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:34.614394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e1d932-4bb8-4927-a0d9-9e307e1808f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:34.680289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d1e1d932-4bb8-4927-a0d9-9e307e1808f4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:34.682307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d1e1d932-4bb8-4927-a0d9-9e307e1808f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:34.690541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:34.779963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:34.818321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2274930-b3d7-4a84-a7b6-01b6f63c1e5b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:34.819910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d2274930-b3d7-4a84-a7b6-01b6f63c1e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:34.925430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2274930-b3d7-4a84-a7b6-01b6f63c1e5b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:34.979879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:34.998386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:34.999000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:34.999318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:34.999597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:35.001365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:35.108006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:35.110036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011a9a3-36e2-4676-a4e6-4e0387392af4 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:35.128050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 733/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:37.177712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d6844-a4d4-415d-8da2-3786d89802b4 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:37.178276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d6844-a4d4-415d-8da2-3786d89802b4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:37.178276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d6844-a4d4-415d-8da2-3786d89802b4 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:37.179642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:37.185136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9ae2-98f1-433f-bac1-66c6a59ff889 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:37.185560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9ae2-98f1-433f-bac1-66c6a59ff889 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:37.185685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9ae2-98f1-433f-bac1-66c6a59ff889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:37.251948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8e6b9ae2-98f1-433f-bac1-66c6a59ff889 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:37.253145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8e6b9ae2-98f1-433f-bac1-66c6a59ff889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:37.290710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:37.348013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:37.385215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d6844-a4d4-415d-8da2-3786d89802b4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:37.386790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ee4d6844-a4d4-415d-8da2-3786d89802b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:37.487525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d6844-a4d4-415d-8da2-3786d89802b4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:37.524072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:37.559623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:37.560217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:37.560490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:37.560754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:37.562329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:37.668037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:37.669468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0fac9-01f4-4573-bb1b-e03a284a2eef service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:37.689465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 730/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:39.772519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa338e-229a-4048-8550-c9f4847a2b79 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:39.773044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa338e-229a-4048-8550-c9f4847a2b79 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:39.773127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa338e-229a-4048-8550-c9f4847a2b79 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:39.774450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:39.780032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f20fd4-b2fe-46c5-8316-61081569adf3 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:39.780490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f20fd4-b2fe-46c5-8316-61081569adf3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:39.780585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f20fd4-b2fe-46c5-8316-61081569adf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:39.848341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-94f20fd4-b2fe-46c5-8316-61081569adf3 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:39.849551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-94f20fd4-b2fe-46c5-8316-61081569adf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:39.867626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:39.945770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:39.982476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa338e-229a-4048-8550-c9f4847a2b79 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:39.984727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-52aa338e-229a-4048-8550-c9f4847a2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:40.087851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa338e-229a-4048-8550-c9f4847a2b79 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:40.152520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:40.159889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:40.164668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:40.164668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:40.164668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:40.164668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:40.271902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:40.273323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8691ff-0a46-43e0-a6a7-cdc50af42358 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:40.291675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 736/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:42.333763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb1486-0f81-49b9-b4b3-644685e0ac49 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:42.334270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb1486-0f81-49b9-b4b3-644685e0ac49 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:42.334270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb1486-0f81-49b9-b4b3-644685e0ac49 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:42.335611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:26:42.340832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f10be-78a4-473c-835c-81a504d5b18f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:42.341279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f10be-78a4-473c-835c-81a504d5b18f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:42.341389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f10be-78a4-473c-835c-81a504d5b18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:42.407764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-907f10be-78a4-473c-835c-81a504d5b18f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:26:42.408954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-907f10be-78a4-473c-835c-81a504d5b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:42.455196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:42.504548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:26:42.541448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb1486-0f81-49b9-b4b3-644685e0ac49 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:26:42.543028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9cfb1486-0f81-49b9-b4b3-644685e0ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:42.646707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb1486-0f81-49b9-b4b3-644685e0ac49 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:26:42.695362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:42.722043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:26:42.722641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:26:42.722913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:26:42.723175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:26:42.724778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:26:42.832008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:26:42.833462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd363816-ea9f-4bb3-b7c8-0a17272457d6 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:26:42.853982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 733/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:26:52.502559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=22 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:26:52.850568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=66 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:02.546601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e3065-7a3b-4410-b5df-87be45b6c5c0 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:02.547129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e3065-7a3b-4410-b5df-87be45b6c5c0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:02.547129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e3065-7a3b-4410-b5df-87be45b6c5c0 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:02.549126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 738/1471] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:02.554463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff1a86-46f0-433f-b10d-826ffda47cb2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:02.554897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff1a86-46f0-433f-b10d-826ffda47cb2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:02.554999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff1a86-46f0-433f-b10d-826ffda47cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:02.620194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e7ff1a86-46f0-433f-b10d-826ffda47cb2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:02.621393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e7ff1a86-46f0-433f-b10d-826ffda47cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:02.719124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 734/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:02.804490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4fc5-004f-45d6-8459-e2b3e6bb399b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:02.804796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4fc5-004f-45d6-8459-e2b3e6bb399b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:02.805079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c4fc5-004f-45d6-8459-e2b3e6bb399b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:02.806413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 739/1473] 10.222.0.15 () {54 vars in 966 bytes} [Mon Aug 8 00:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:02.811096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9591-f58b-4a48-99bc-7ae7dcfd773d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:02.811409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9591-f58b-4a48-99bc-7ae7dcfd773d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:02.811678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee9591-f58b-4a48-99bc-7ae7dcfd773d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:02.887115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-eaee9591-f58b-4a48-99bc-7ae7dcfd773d None None] MFA Rules not processed for user `de6c52a7640e47c7be2253e32aa5a173`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:02.893459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-eaee9591-f58b-4a48-99bc-7ae7dcfd773d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:02.914129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:02.969203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 735/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Mon Aug 8 00:27:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:02.976936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c4fc5-004f-45d6-8459-e2b3e6bb399b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:02.978458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a12c4fc5-004f-45d6-8459-e2b3e6bb399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:03.081395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c4fc5-004f-45d6-8459-e2b3e6bb399b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:03.152447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:03.153043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:03.153319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:03.153587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:03.155164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:03.208498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:03.262028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:03.263517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83b0c31-cae6-4016-aab0-c322265fdc5f service glance] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:03.281342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 740/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:27:05.377205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569c0b3-b70a-4075-a9fc-95f88a80be0d None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:05.377790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569c0b3-b70a-4075-a9fc-95f88a80be0d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:05.377790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569c0b3-b70a-4075-a9fc-95f88a80be0d None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:05.379085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 736/1476] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:05.384207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee745db-2c32-4d53-b35d-1f26b9baff1c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:05.384635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee745db-2c32-4d53-b35d-1f26b9baff1c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:05.384760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee745db-2c32-4d53-b35d-1f26b9baff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:05.450894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-fee745db-2c32-4d53-b35d-1f26b9baff1c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:05.452116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fee745db-2c32-4d53-b35d-1f26b9baff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:05.545559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 741/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:05.570122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569c0b3-b70a-4075-a9fc-95f88a80be0d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:05.571676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7569c0b3-b70a-4075-a9fc-95f88a80be0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:05.695521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569c0b3-b70a-4075-a9fc-95f88a80be0d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:05.743325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:05.786642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:05.787250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:05.787535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:05.787800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:05.788423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:05.790128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:27:05.790128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:27:05.790128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:27:05.791281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b9029-931d-4a55-8ad0-28e786252eb5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:05.809948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 737/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Mon Aug 8 00:27:05 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:27:07.892572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f0760-1daa-4d1d-91d6-e5130e43528f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:07.893163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f0760-1daa-4d1d-91d6-e5130e43528f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:07.893163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f0760-1daa-4d1d-91d6-e5130e43528f None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:07.894459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 742/1479] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:07.899692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c675b96-95ad-42c8-9d15-0b382b06022f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:07.900095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c675b96-95ad-42c8-9d15-0b382b06022f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:07.900246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c675b96-95ad-42c8-9d15-0b382b06022f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:07.966632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8c675b96-95ad-42c8-9d15-0b382b06022f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:07.967781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8c675b96-95ad-42c8-9d15-0b382b06022f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:08.062589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 738/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:08.106741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f0760-1daa-4d1d-91d6-e5130e43528f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:08.108252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ca7f0760-1daa-4d1d-91d6-e5130e43528f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:08.119106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:08.252239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f0760-1daa-4d1d-91d6-e5130e43528f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:08.330398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:08.363414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:08.364013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:08.364289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:08.364564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:08.366155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:08.452621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:08.454723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d91fef-7ae7-4fac-a0d5-ff04ad2f0a0d service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:08.472729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 743/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:27:10.648678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a9407-95ae-4286-8d0e-f504660413bf None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:10.649210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a9407-95ae-4286-8d0e-f504660413bf None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:10.649210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a9407-95ae-4286-8d0e-f504660413bf None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:10.650754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 739/1482] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:10.656008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8afe1-9c70-439a-8c82-0cf7262dbf04 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:10.656417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8afe1-9c70-439a-8c82-0cf7262dbf04 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:10.656549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8afe1-9c70-439a-8c82-0cf7262dbf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:10.722474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-08f8afe1-9c70-439a-8c82-0cf7262dbf04 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:10.723710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-08f8afe1-9c70-439a-8c82-0cf7262dbf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:10.770468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:10.824639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 744/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:10.862174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a9407-95ae-4286-8d0e-f504660413bf None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:10.863730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c66a9407-95ae-4286-8d0e-f504660413bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:10.887281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:10.966405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a9407-95ae-4286-8d0e-f504660413bf None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:11.037706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:11.038310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:11.038587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:11.038853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:11.040502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:11.113209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:11.147967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:11.149547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ffe19-5dba-45ae-b7c0-353a234a42c4 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:11.167707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 740/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:27:13.217569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7de5a-7bd2-4ccf-bb38-7fda6e79cb6b None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:13.218151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7de5a-7bd2-4ccf-bb38-7fda6e79cb6b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:13.218151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7de5a-7bd2-4ccf-bb38-7fda6e79cb6b None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:13.219995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 745/1485] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:13.225049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3ec63-e3a2-4ce2-ae44-ae42cd93263d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:13.225462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3ec63-e3a2-4ce2-ae44-ae42cd93263d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:13.225585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3ec63-e3a2-4ce2-ae44-ae42cd93263d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:13.290610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cfb3ec63-e3a2-4ce2-ae44-ae42cd93263d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:13.291838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cfb3ec63-e3a2-4ce2-ae44-ae42cd93263d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:13.392884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 741/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:13.431310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7de5a-7bd2-4ccf-bb38-7fda6e79cb6b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:13.432905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-96f7de5a-7bd2-4ccf-bb38-7fda6e79cb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:13.535258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7de5a-7bd2-4ccf-bb38-7fda6e79cb6b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:13.605561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:13.607260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:13.607932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:13.608208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:13.608475 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:13.610681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:13.720072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:13.721503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa196e13-7acc-41d9-a3ac-0d1c867b026f service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:13.739640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 746/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:27:15.949943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c035e-b72f-4dee-b434-b568f6f85ac4 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:15.950454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c035e-b72f-4dee-b434-b568f6f85ac4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:15.950454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c035e-b72f-4dee-b434-b568f6f85ac4 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:15.952065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 742/1488] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:15.957426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c816f9-e457-4100-a5fb-25650b45ce26 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:15.957850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c816f9-e457-4100-a5fb-25650b45ce26 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:15.957980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c816f9-e457-4100-a5fb-25650b45ce26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:16.022451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-86c816f9-e457-4100-a5fb-25650b45ce26 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:16.023672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-86c816f9-e457-4100-a5fb-25650b45ce26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:16.074395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:16.119445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 747/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:16.160513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c035e-b72f-4dee-b434-b568f6f85ac4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:16.162099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ee3c035e-b72f-4dee-b434-b568f6f85ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:16.197707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:16.265661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c035e-b72f-4dee-b434-b568f6f85ac4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:16.340209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:16.340808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:16.341089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:16.341364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:16.342966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:16.423973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:16.452273 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:16.454213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92d577-8958-405f-89a9-af1975443e6d service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:16.474995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 743/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:27:18.898217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a32038-489f-4b7e-9563-fac2727bfe3f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:18.898739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a32038-489f-4b7e-9563-fac2727bfe3f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:18.898739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a32038-489f-4b7e-9563-fac2727bfe3f None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:18.900120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 748/1491] 10.222.0.15 () {54 vars in 927 bytes} [Mon Aug 8 00:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:27:18.905287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd5221-fce6-4934-be43-9bbc7cb8d483 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:18.905692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd5221-fce6-4934-be43-9bbc7cb8d483 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:18.905832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd5221-fce6-4934-be43-9bbc7cb8d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:18.972803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f2fd5221-fce6-4934-be43-9bbc7cb8d483 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:27:18.974095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f2fd5221-fce6-4934-be43-9bbc7cb8d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:19.068949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 744/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Mon Aug 8 00:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:27:19.109889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a32038-489f-4b7e-9563-fac2727bfe3f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:27:19.111451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-28a32038-489f-4b7e-9563-fac2727bfe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:19.213245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a32038-489f-4b7e-9563-fac2727bfe3f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:27:19.288418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:27:19.289101 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:27:19.289424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:27:19.289701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:27:19.291298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:27:19.300691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:19.398195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:27:19.399685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3575ec6-9c70-43f7-a048-c0f1f9140a29 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:27:19.418549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 749/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:27:29.067306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=82 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:27:29.416255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=50 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:52.012078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:52.012636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:52.012636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:52.087775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:28:52.089012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:52.188208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 745/1494] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:28:52.196682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:52.197210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:52.197210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:52.273761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:28:52.274930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:52.371157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 750/1495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:28:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:28:52.375885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:52.377132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:52.389034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f83b3a-30e3-4d9f-8870-6465c6d91ba9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:52.449873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:52.483182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:52.483816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:52.484096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:52.484368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:52.485003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:28:52.485304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:52.486398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:52.486398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:28:52.486398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:28:52.487311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:52.498469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 746/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:28:52.507292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:52.516280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8b4619-49fe-4e70-ba1c-62e43674a7d0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:52.578370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:52.610972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:28:52.627921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:28:52.628610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:52.716730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 751/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:28:52 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:28:52.721055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:52.722587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:52.755608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826f8f2-342c-4e43-a4f2-122659891e4e admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:52.849687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:52.850322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:52.850597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:52.850872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:52.851537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:52.852556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:52.852556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:28:52.852556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:28:52.853744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:52.944705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 747/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:28:52 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:28:52.948935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:52.950480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:53.011945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4968f3-3ba1-4553-aabf-0388ad12d7b9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:53.092686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:53.106951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:53.107523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:53.107818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:53.108089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:53.108726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:53.109754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:53.109754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:28:53.109754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:28:53.110631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:53.124295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 752/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:28:53.127735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:53.129315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:53.140944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6f33b-79db-4a2e-aacb-4b11b27faa38 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:53.165916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:53.234702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:53.235371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:53.235648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:53.235921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:53.236559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:53.237571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:53.237571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:28:53.237571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:28:53.238456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1e70d-4273-4d78-bbd7-a0ad53140795 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:53.251052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 748/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:28:53.255333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:53.256944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:53.290008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9d7f9-5717-48d7-9b81-d0f3253896ae admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:53.382718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:53.383323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:53.383603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:53.383870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] PATH_INFO: `/v3/projects/815dfc41e7e04b66932eaec4f75f7586/users/1bae95d4afd14c48a1a0f683bf5edc3b/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:53.405861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=815dfc41e7e04b66932eaec4f75f7586, user_id=1bae95d4afd14c48a1a0f683bf5edc3b, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:53.406944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:53.406944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:28:53.406944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:28:53.408926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1071c3d-5264-4804-b301-f583b485cb78 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:53.527406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 753/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:28:53 2022] PUT /identity/v3/projects/815dfc41e7e04b66932eaec4f75f7586/users/1bae95d4afd14c48a1a0f683bf5edc3b/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:28:53.539978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5d81-5731-4509-ad58-fad24599099c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:53.540261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5d81-5731-4509-ad58-fad24599099c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:53.540529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cda5d81-5731-4509-ad58-fad24599099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:53.596648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7cda5d81-5731-4509-ad58-fad24599099c None None] MFA Rules not processed for user `1bae95d4afd14c48a1a0f683bf5edc3b`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:28:53.597842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7cda5d81-5731-4509-ad58-fad24599099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:53.674045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 749/1502] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:28:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:28:53.687260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642194e4-9231-443c-8929-b94173ae854f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:53.687533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642194e4-9231-443c-8929-b94173ae854f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:53.687807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642194e4-9231-443c-8929-b94173ae854f None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:53.689137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 754/1503] 10.222.0.15 () {54 vars in 964 bytes} [Mon Aug 8 00:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:28:53.693911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21056eb-dd9b-40e1-aebb-847ece6f47d4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:53.694195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21056eb-dd9b-40e1-aebb-847ece6f47d4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:53.694468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21056eb-dd9b-40e1-aebb-847ece6f47d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:53.726310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:53.769031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c21056eb-dd9b-40e1-aebb-847ece6f47d4 None None] MFA Rules not processed for user `2e66698136f4429b8df67e5050a8da2f`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:28:53.770226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c21056eb-dd9b-40e1-aebb-847ece6f47d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:53.845001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 750/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Mon Aug 8 00:28:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:28:53.853043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642194e4-9231-443c-8929-b94173ae854f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:53.854587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-642194e4-9231-443c-8929-b94173ae854f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:53.905144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:53.959407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642194e4-9231-443c-8929-b94173ae854f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:54.030783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:54.031409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:54.031691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:54.031962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:54.033556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:54.129654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:54.131633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:54.135112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:54.149937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 755/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:28:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:28:54.194713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d00be6-849e-497d-85cc-075ddd03252e None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:54.195645 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d00be6-849e-497d-85cc-075ddd03252e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:54.195862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d00be6-849e-497d-85cc-075ddd03252e None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:54.197141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 751/1506] 10.222.0.15 () {52 vars in 830 bytes} [Mon Aug 8 00:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:28:54.200590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:54.201827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:54.298934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b1e66-33c8-422f-92ff-0cf1480e7a49 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:54.390570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:54.391186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:54.391493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:54.391759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] PATH_INFO: `/v3/users/1bae95d4afd14c48a1a0f683bf5edc3b` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:54.403532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bae95d4afd14c48a1a0f683bf5edc3b)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:54.404543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:54.404543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:28:54.404543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:28:54.405734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:54.415116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:54.602707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bae95d4afd14c48a1a0f683bf5edc3b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:54.625753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bae95d4afd14c48a1a0f683bf5edc3b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:54.626365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bae95d4afd14c48a1a0f683bf5edc3b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:54.641502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bae95d4afd14c48a1a0f683bf5edc3b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:54.742317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bae95d4afd14c48a1a0f683bf5edc3b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:54.748358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 756/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:28:54 2022] DELETE /identity/v3/users/1bae95d4afd14c48a1a0f683bf5edc3b => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:28:54.752771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:54.753060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:54.753334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:54.810247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:28:54.811485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:54.886729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:54.899631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 752/1508] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:28:54.909498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:54.910712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:55.010044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e58d6-1b3e-407c-81d4-326dd79bec4e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:55.063702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:28:55.082271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:55.082877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:55.083156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:55.083456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:55.085789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:55.192036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:55.193415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd17372-a0ce-43e6-81d7-b54a06f42515 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:55.211946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 757/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:28:55.654979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:28:55.656239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:28:55.670534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e770b35-dad2-4dd0-849a-2f9360e5de45 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:28:55.765656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:28:55.766290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:28:55.766568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:28:55.766838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] PATH_INFO: `/v3/projects/815dfc41e7e04b66932eaec4f75f7586` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:28:55.776700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=815dfc41e7e04b66932eaec4f75f7586)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:28:55.777729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:28:55.777729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:28:55.777729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:28:55.778925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:28:55.890103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '815dfc41e7e04b66932eaec4f75f7586', 'request_id': 'req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:55.896040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '815dfc41e7e04b66932eaec4f75f7586', 'request_id': 'req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:55.909218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '815dfc41e7e04b66932eaec4f75f7586', 'request_id': 'req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:56.011063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] The token cache is being invalidate because project 815dfc41e7e04b66932eaec4f75f7586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:28:56.011281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:56.011812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-162ad55c-7dd9-45a4-8b51-3e06b603fa8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:28:56.013112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 753/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:28:55 2022] DELETE /identity/v3/projects/815dfc41e7e04b66932eaec4f75f7586 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:29:05.209741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=62 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:29:06.009008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:30:24.311220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbf884-5839-41f1-9ab7-f0f1a4118b1d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:24.311928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbf884-5839-41f1-9ab7-f0f1a4118b1d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:24.311928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbf884-5839-41f1-9ab7-f0f1a4118b1d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:24.313128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:30:24.318075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a242875-d35c-4c34-a39e-d64fa1e4692d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:24.318505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a242875-d35c-4c34-a39e-d64fa1e4692d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:24.318612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a242875-d35c-4c34-a39e-d64fa1e4692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:24.397306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9a242875-d35c-4c34-a39e-d64fa1e4692d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:24.398540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9a242875-d35c-4c34-a39e-d64fa1e4692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:24.501953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:30:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:24.653409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbf884-5839-41f1-9ab7-f0f1a4118b1d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:24.655605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9cbbf884-5839-41f1-9ab7-f0f1a4118b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:24.774804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbf884-5839-41f1-9ab7-f0f1a4118b1d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:24.780908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9403b9-7ed4-496d-a174-8777d06e34a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:24.781503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9403b9-7ed4-496d-a174-8777d06e34a2 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:24.781781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9403b9-7ed4-496d-a174-8777d06e34a2 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:24.782050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9403b9-7ed4-496d-a174-8777d06e34a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:24.790214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9403b9-7ed4-496d-a174-8777d06e34a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:24.792512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9403b9-7ed4-496d-a174-8777d06e34a2 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:24.801755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 759/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:25.638799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a04ff9-5f60-4365-a572-59e29543b4f7 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:25.639397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a04ff9-5f60-4365-a572-59e29543b4f7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:25.639397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a04ff9-5f60-4365-a572-59e29543b4f7 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:25.640660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:25.645682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d317e51-a104-4ac7-b24e-f4c83ad7b295 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:25.646082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d317e51-a104-4ac7-b24e-f4c83ad7b295 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:25.646234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d317e51-a104-4ac7-b24e-f4c83ad7b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:25.666952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-9d317e51-a104-4ac7-b24e-f4c83ad7b295 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:25.668194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9d317e51-a104-4ac7-b24e-f4c83ad7b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:25.676083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:30:25.827798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a04ff9-5f60-4365-a572-59e29543b4f7 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:25.843245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a04ff9-5f60-4365-a572-59e29543b4f7 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:25.848706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a91d6-063f-4020-8223-9118d7f63eee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:25.849306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a91d6-063f-4020-8223-9118d7f63eee service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:25.849584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a91d6-063f-4020-8223-9118d7f63eee service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:25.849856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a91d6-063f-4020-8223-9118d7f63eee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:25.858914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a91d6-063f-4020-8223-9118d7f63eee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:25.860534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a91d6-063f-4020-8223-9118d7f63eee service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:25.869882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 756/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:26.409045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe859f6-a63a-4913-8ec9-507743e0521f None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:26.409333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe859f6-a63a-4913-8ec9-507743e0521f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:26.409607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe859f6-a63a-4913-8ec9-507743e0521f None None] PATH_INFO: `/v3` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:26.410898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 761/1517] 10.222.0.25 () {54 vars in 925 bytes} [Mon Aug 8 00:30:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:30:26.417503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cafaf8-4815-48c2-b6aa-a4337ec4e8a6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:26.417788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cafaf8-4815-48c2-b6aa-a4337ec4e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:26.418064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cafaf8-4815-48c2-b6aa-a4337ec4e8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:26.476692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-92cafaf8-4815-48c2-b6aa-a4337ec4e8a6 None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:26.477895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-92cafaf8-4815-48c2-b6aa-a4337ec4e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:26.494746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:30:26.548116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 757/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Aug 8 00:30:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:30.604932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac8445-594c-4269-8aac-1286d451c9e1 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:30.605498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac8445-594c-4269-8aac-1286d451c9e1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:30.605498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac8445-594c-4269-8aac-1286d451c9e1 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:30.607011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 762/1519] 10.222.0.15 () {54 vars in 961 bytes} [Mon Aug 8 00:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:30.612086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a6493-ab19-4162-8f6a-466f344e0fc9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:30.613501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a6493-ab19-4162-8f6a-466f344e0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:30.613501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3a6493-ab19-4162-8f6a-466f344e0fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:30.631951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bb3a6493-ab19-4162-8f6a-466f344e0fc9 None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:30.634458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bb3a6493-ab19-4162-8f6a-466f344e0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:30.642541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 758/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Aug 8 00:30:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:30.650073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac8445-594c-4269-8aac-1286d451c9e1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:30.664076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac8445-594c-4269-8aac-1286d451c9e1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:30.669382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f9f71-8473-4671-9fdb-6841da1f58f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:30.669991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9f71-8473-4671-9fdb-6841da1f58f3 service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:30.670273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9f71-8473-4671-9fdb-6841da1f58f3 service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:30.670593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f9f71-8473-4671-9fdb-6841da1f58f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:30.679497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f9f71-8473-4671-9fdb-6841da1f58f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:30.681078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f9f71-8473-4671-9fdb-6841da1f58f3 service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:30.691245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 763/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:37.113177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2904-731e-42d4-af23-7a74126574c2 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:37.113706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2904-731e-42d4-af23-7a74126574c2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:37.113706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded2904-731e-42d4-af23-7a74126574c2 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:37.115067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:37.119893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78576-4dd4-43fb-b100-2b005b36156b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:37.120327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78576-4dd4-43fb-b100-2b005b36156b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:37.120438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78576-4dd4-43fb-b100-2b005b36156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:37.141098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-75b78576-4dd4-43fb-b100-2b005b36156b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:37.142344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-75b78576-4dd4-43fb-b100-2b005b36156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:37.150099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:30:37.302879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2904-731e-42d4-af23-7a74126574c2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:37.317463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded2904-731e-42d4-af23-7a74126574c2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:37.322794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d298e-84f8-4ae3-86cc-176aa7015e66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:37.329717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d298e-84f8-4ae3-86cc-176aa7015e66 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:37.329717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d298e-84f8-4ae3-86cc-176aa7015e66 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:37.329717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d298e-84f8-4ae3-86cc-176aa7015e66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:37.332635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d298e-84f8-4ae3-86cc-176aa7015e66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:37.334197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d298e-84f8-4ae3-86cc-176aa7015e66 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:37.344049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 760/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:38.162187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14087b-5a30-4a45-b1e8-50b25cef3755 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:38.162730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14087b-5a30-4a45-b1e8-50b25cef3755 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:38.162730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb14087b-5a30-4a45-b1e8-50b25cef3755 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:38.164104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 00:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:38.168746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50f20-0678-461a-b780-f325921b3070 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:38.169200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50f20-0678-461a-b780-f325921b3070 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:38.170019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50f20-0678-461a-b780-f325921b3070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:38.188683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-22a50f20-0678-461a-b780-f325921b3070 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:38.189861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-22a50f20-0678-461a-b780-f325921b3070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:38.197520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 00:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:30:38.351367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14087b-5a30-4a45-b1e8-50b25cef3755 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:38.366163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb14087b-5a30-4a45-b1e8-50b25cef3755 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:38.371715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95406b-38ab-4f95-b8cd-9abbf24b8231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:38.372303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95406b-38ab-4f95-b8cd-9abbf24b8231 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:38.372892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95406b-38ab-4f95-b8cd-9abbf24b8231 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:38.373165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95406b-38ab-4f95-b8cd-9abbf24b8231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:38.381291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95406b-38ab-4f95-b8cd-9abbf24b8231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:38.382852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95406b-38ab-4f95-b8cd-9abbf24b8231 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:38.392132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 766/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:42.048335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4350a4-2ba1-4d91-9b26-09323942225e None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:42.048896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4350a4-2ba1-4d91-9b26-09323942225e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:42.048896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4350a4-2ba1-4d91-9b26-09323942225e None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:42.050209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:42.055547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f54c-3514-4452-bd1c-3b36237569e1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:42.056018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f54c-3514-4452-bd1c-3b36237569e1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:42.056131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f54c-3514-4452-bd1c-3b36237569e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:42.076712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-72d1f54c-3514-4452-bd1c-3b36237569e1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:42.077930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-72d1f54c-3514-4452-bd1c-3b36237569e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:42.086413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:42.161774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4350a4-2ba1-4d91-9b26-09323942225e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:42.163759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5c4350a4-2ba1-4d91-9b26-09323942225e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:42.242842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4350a4-2ba1-4d91-9b26-09323942225e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:42.246484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff702008-e69d-4ff5-ab90-af438eccff56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:42.247083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702008-e69d-4ff5-ab90-af438eccff56 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:42.248006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702008-e69d-4ff5-ab90-af438eccff56 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:42.248285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff702008-e69d-4ff5-ab90-af438eccff56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:42.257212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702008-e69d-4ff5-ab90-af438eccff56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:42.258938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff702008-e69d-4ff5-ab90-af438eccff56 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:42.268888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 763/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:44.273448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a5a2-ee48-4365-91c3-23c9c52b974d None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:44.274015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a5a2-ee48-4365-91c3-23c9c52b974d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:44.274015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a5a2-ee48-4365-91c3-23c9c52b974d None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:44.275483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:30:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:44.281233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb540b-d2e4-4170-a32f-7b68fd139a2e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:44.281679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb540b-d2e4-4170-a32f-7b68fd139a2e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:44.281778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb540b-d2e4-4170-a32f-7b68fd139a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:44.301436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9edb540b-d2e4-4170-a32f-7b68fd139a2e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:44.302614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9edb540b-d2e4-4170-a32f-7b68fd139a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:44.310419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:30:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:44.346123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a5a2-ee48-4365-91c3-23c9c52b974d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:44.357294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a5a2-ee48-4365-91c3-23c9c52b974d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:44.360832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5071413b-103f-4c42-906f-cb3830658563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:44.361411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071413b-103f-4c42-906f-cb3830658563 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:44.361684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071413b-103f-4c42-906f-cb3830658563 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:44.361954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071413b-103f-4c42-906f-cb3830658563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:44.370611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071413b-103f-4c42-906f-cb3830658563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:44.372053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5071413b-103f-4c42-906f-cb3830658563 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:44.381680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 769/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:30:47.875847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc887a-aaf9-4ab4-8d99-ce5ae7311140 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:47.876384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc887a-aaf9-4ab4-8d99-ce5ae7311140 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:47.876530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc887a-aaf9-4ab4-8d99-ce5ae7311140 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:47.877854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:47.883043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55110a-3920-4eeb-806c-d8aee4180a42 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:47.883494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55110a-3920-4eeb-806c-d8aee4180a42 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:47.883622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55110a-3920-4eeb-806c-d8aee4180a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:47.891312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=81 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:30:47.903042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4b55110a-3920-4eeb-806c-d8aee4180a42 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:47.904867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4b55110a-3920-4eeb-806c-d8aee4180a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:47.912697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:48.220732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc887a-aaf9-4ab4-8d99-ce5ae7311140 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:48.222781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a3dc887a-aaf9-4ab4-8d99-ce5ae7311140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:48.288076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:30:48.300951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc887a-aaf9-4ab4-8d99-ce5ae7311140 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:48.304878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e1223-c99a-4a53-8c43-a76ec7459a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:48.305542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e1223-c99a-4a53-8c43-a76ec7459a3d service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:48.305823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e1223-c99a-4a53-8c43-a76ec7459a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:48.306096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e1223-c99a-4a53-8c43-a76ec7459a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:48.315610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e1223-c99a-4a53-8c43-a76ec7459a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:48.317043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e1223-c99a-4a53-8c43-a76ec7459a3d service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:48.327765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 766/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:30:50.360171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f416c5-74e8-427d-8f67-7a4400c8dc8c None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:50.360758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f416c5-74e8-427d-8f67-7a4400c8dc8c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:50.360758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f416c5-74e8-427d-8f67-7a4400c8dc8c None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:50.362048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 00:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:30:50.367155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287a90b-f0e8-4b9c-8039-fda411d7f0d0 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:50.367584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287a90b-f0e8-4b9c-8039-fda411d7f0d0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:50.367741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287a90b-f0e8-4b9c-8039-fda411d7f0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:50.387282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3287a90b-f0e8-4b9c-8039-fda411d7f0d0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:30:50.389003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3287a90b-f0e8-4b9c-8039-fda411d7f0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:50.396664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 00:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:30:50.697658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f416c5-74e8-427d-8f67-7a4400c8dc8c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:30:50.699450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b2f416c5-74e8-427d-8f67-7a4400c8dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:30:50.712790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f416c5-74e8-427d-8f67-7a4400c8dc8c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:30:50.716391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa078-80dd-4a10-bdd3-e81ca2c9cbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:30:50.717021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa078-80dd-4a10-bdd3-e81ca2c9cbee service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:30:50.717298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa078-80dd-4a10-bdd3-e81ca2c9cbee service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:30:50.717566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa078-80dd-4a10-bdd3-e81ca2c9cbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:30:50.726400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa078-80dd-4a10-bdd3-e81ca2c9cbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:30:50.727852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa078-80dd-4a10-bdd3-e81ca2c9cbee service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:30:50.737684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 772/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:31:00.382719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=11 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:31:00.733529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=8 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:14.905877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:14.907504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:14.907956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:14.933655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:14.934878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:14.943599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 768/1540] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:14.952101 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:14.952669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:14.952669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:14.984547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:14.986369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.001386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 773/1541] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.006219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.017436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e863ab-f14a-46bf-8fc8-5464b91f0213 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.027846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.028482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.028761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.029033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.029689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:15.029992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.032166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f657d2-e241-42eb-9859-bbd8a43d266d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.044846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 769/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:15.053573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.073845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86816c2-d99d-4a71-b6f3-4bb3cc5c84e3 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.084130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511e45e-95a0-4100-b029-61271a4959c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.084772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e45e-95a0-4100-b029-61271a4959c7 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.085054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e45e-95a0-4100-b029-61271a4959c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.085329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511e45e-95a0-4100-b029-61271a4959c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.085994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e45e-95a0-4100-b029-61271a4959c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.088056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511e45e-95a0-4100-b029-61271a4959c7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.226367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.226656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.226845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.287746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 774/1543] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:15.289064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070d52c-f75d-4b28-a878-9d084d595a58 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.289353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070d52c-f75d-4b28-a878-9d084d595a58 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.289630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070d52c-f75d-4b28-a878-9d084d595a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.293196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.294463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.328311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b070d52c-f75d-4b28-a878-9d084d595a58 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.329873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b070d52c-f75d-4b28-a878-9d084d595a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.428217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 770/1544] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 08 00:32:15.429214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.447156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd243691-cfd7-4189-84bb-9af16685527f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.456678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba04e0-6d87-49c1-886a-0c68d30642ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.469744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba04e0-6d87-49c1-886a-0c68d30642ca admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.469744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba04e0-6d87-49c1-886a-0c68d30642ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.469744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba04e0-6d87-49c1-886a-0c68d30642ca admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.469744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba04e0-6d87-49c1-886a-0c68d30642ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.469744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba04e0-6d87-49c1-886a-0c68d30642ca admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.470269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 775/1545] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.470401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d318f83-3ac7-406b-b200-a76e12838851 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.470637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d318f83-3ac7-406b-b200-a76e12838851 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.470916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d318f83-3ac7-406b-b200-a76e12838851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.504243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8d318f83-3ac7-406b-b200-a76e12838851 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.505412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8d318f83-3ac7-406b-b200-a76e12838851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.514366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 776/1546] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.516410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944592-0f0e-408b-be64-74389efde6a9 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.516752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944592-0f0e-408b-be64-74389efde6a9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.517074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78944592-0f0e-408b-be64-74389efde6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.540400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-78944592-0f0e-408b-be64-74389efde6a9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.541610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-78944592-0f0e-408b-be64-74389efde6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.549586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 777/1547] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.550757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18497da-11a7-43d0-afba-216088cace24 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.551039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18497da-11a7-43d0-afba-216088cace24 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.551331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18497da-11a7-43d0-afba-216088cace24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.556607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 771/1548] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:15.559701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.559701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.559701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.572254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b18497da-11a7-43d0-afba-216088cace24 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.573515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b18497da-11a7-43d0-afba-216088cace24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.578272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.582923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.586322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 778/1549] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.587591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875fef7-afff-444d-8f60-8fb4b7a40aba None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.587875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875fef7-afff-444d-8f60-8fb4b7a40aba None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.588148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875fef7-afff-444d-8f60-8fb4b7a40aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.589123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 772/1550] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.589987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.599980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6cabd-70b0-43a7-95d6-187d36019a96 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.605935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.607580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.607580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.607580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.607932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:15.608256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.609919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d5e5c-def3-4775-b391-1caf743ff258 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.611287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-9875fef7-afff-444d-8f60-8fb4b7a40aba None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.612489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9875fef7-afff-444d-8f60-8fb4b7a40aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.620518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 779/1551] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.621698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.627309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.627469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 773/1552] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:15.627571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.627676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2873-8e5d-42bc-949f-0b8e868d3005 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.627676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2873-8e5d-42bc-949f-0b8e868d3005 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.627676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2873-8e5d-42bc-949f-0b8e868d3005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.644467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c56b2873-8e5d-42bc-949f-0b8e868d3005 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.645642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.645797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c56b2873-8e5d-42bc-949f-0b8e868d3005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.646836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.653929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 774/1553] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.655023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.655141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 780/1554] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.664903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.664903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.665122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.671502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549fda3-2758-4bea-9a1c-8ef77c1b23d7 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.677134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c15d21-2da1-4e20-a208-a2c0b4726819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.677762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15d21-2da1-4e20-a208-a2c0b4726819 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.678084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15d21-2da1-4e20-a208-a2c0b4726819 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.678354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.678471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c15d21-2da1-4e20-a208-a2c0b4726819 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.678995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c15d21-2da1-4e20-a208-a2c0b4726819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.679624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.680640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c15d21-2da1-4e20-a208-a2c0b4726819 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.688409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 775/1555] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.689424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.691730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 781/1556] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:15.693598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb512e-9911-463d-9f55-52467acc62c7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.693598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb512e-9911-463d-9f55-52467acc62c7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.693598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb512e-9911-463d-9f55-52467acc62c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.705317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92857dfc-d216-4f80-afdb-099ed496a060 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.710430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.711039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.711503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.711734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.712340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:15.712645 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.714390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.714890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d5bb512e-9911-463d-9f55-52467acc62c7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.716463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d5bb512e-9911-463d-9f55-52467acc62c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.725633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 782/1557] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.726828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390ed7-eb16-4a4b-8150-c0db5b339ebb None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.727112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390ed7-eb16-4a4b-8150-c0db5b339ebb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.727416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec390ed7-eb16-4a4b-8150-c0db5b339ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.746720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ec390ed7-eb16-4a4b-8150-c0db5b339ebb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.747936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ec390ed7-eb16-4a4b-8150-c0db5b339ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.756858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 776/1558] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:15.757705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.758117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 783/1559] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.759391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72e365-d3df-4b66-a0ed-ed8b10e4d2ed None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.759671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72e365-d3df-4b66-a0ed-ed8b10e4d2ed None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.759939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72e365-d3df-4b66-a0ed-ed8b10e4d2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.776126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa54b88-00f1-420b-8585-e55921576abc admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.780720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.782242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.782242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.782451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.783076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.783609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ca72e365-d3df-4b66-a0ed-ed8b10e4d2ed None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.784195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:32:15.784195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:32:15.784195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:32:15.784799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ca72e365-d3df-4b66-a0ed-ed8b10e4d2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.785372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.792559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 784/1560] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.793777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.794121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.794392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.814248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:15.821004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:15.823529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 785/1561] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:15.824301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.833855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191da0b5-d130-49ca-bae2-d7bdfff55d9f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.838427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.839001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.839314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.839648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.840276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:15.840576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.841619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:32:15.841619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:32:15.841619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:32:15.842495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.854504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 786/1562] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:15.862418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.865228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff4d9e4-4cb1-4173-9857-e1634b07d979 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.869841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.870416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.870686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.870954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.871598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.873188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.878799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 777/1563] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:15.878799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.887387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 787/1564] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:15.887387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.895765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a205e-c510-4879-b578-9be3bd5a6e0f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.921588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:15.937172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:15.940881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de713bd-264d-4505-9f8f-a629a4399f9f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.945680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.946277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.946554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.946824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.947479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:15.947778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.949512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.954752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.960146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.960146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.960146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.960146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.960146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.961162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 778/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:15.962040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:15.971751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b6f89-0666-49e9-ace8-b76899bd3567 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:15.976324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:15.976948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:15.977420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:15.977846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:15.978842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:15.979188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:15.980798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:15.993124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 779/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:15.994040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.003714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef74c-6f65-4a96-b9ed-6e3e97a7076f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.008174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.008698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.008971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.009242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.009849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:16.010144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.011705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.022766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 780/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:16.023667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.032962 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bff0e-06b4-4a1b-ba99-bb599848269b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.037610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.038189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.038469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.038739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.039400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:16.039868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.041433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.042609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 788/1568] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:15 2022] POST /identity/v3/projects => generated 391 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.043449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.054552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 781/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:16.055384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.065724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e4ded-267d-4f40-854e-0810953f9aef admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.114859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed089550-78d7-4326-a947-8fb9c6d289b8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.119510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.120112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.120385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.120654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.121025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.121347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.121620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.121895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.122164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.122774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.123278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.124687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.221845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 782/1570] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/users => generated 282 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.222726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.232663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd62356-00ad-4598-94a4-9df846145146 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.238567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 789/1571] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.238567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.294119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.295417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.295417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.295674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] PATH_INFO: `/v3/projects/efcecb80f0c64612a23a742d93b36380/users/dc304004263343378433935147ce2105/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.299415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae4a75c-1215-4a38-8b2e-d187e2b88c4d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.304423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.305022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.309120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.309120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.309120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.309120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.310938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] RBAC: Authorizing `identity:create_grant(project_id=efcecb80f0c64612a23a742d93b36380, user_id=dc304004263343378433935147ce2105, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.312042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:32:16.312042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:32:16.312042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:32:16.314056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.411109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 783/1572] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:16 2022] PUT /identity/v3/projects/efcecb80f0c64612a23a742d93b36380/users/dc304004263343378433935147ce2105/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:16.411942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.436635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 790/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/projects => generated 389 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.437601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.440745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:16.463908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:16.478158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9ee9b-4539-4fd3-8ffd-9089f164424d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.498758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9d7d0-a850-473d-acce-fb1613bc2bfa admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.503542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.506539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.506539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.506539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.506539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.507210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.533841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.534468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.535152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.535152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.535635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.537544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.589493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 791/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.590310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.613297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 784/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/projects => generated 403 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.615492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.657140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795015d3-1f00-4b48-90e0-6ad138748443 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.674424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a70b2-06f6-4268-89bc-03a13b73038d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.679060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.679687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.679965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.680232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.680884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.682780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.718829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.720285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.720566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.720839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.721455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.722461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:32:16.722461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:32:16.722461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:32:16.723703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.772380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 785/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.773243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.815525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:16.835748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c5851-9e3b-4ca0-9783-93be31b71635 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.840430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 792/1577] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/users => generated 296 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.840430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.843207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3966c0-7fb8-43f4-929a-bcd39c250c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.843932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3966c0-7fb8-43f4-929a-bcd39c250c8d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.844966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3966c0-7fb8-43f4-929a-bcd39c250c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.845261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3966c0-7fb8-43f4-929a-bcd39c250c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.845922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3966c0-7fb8-43f4-929a-bcd39c250c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.847601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3966c0-7fb8-43f4-929a-bcd39c250c8d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.851010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d5da1-1ef2-45b2-be4b-6a224183facd admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.855908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.856609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.857024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.857288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.861132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 786/1578] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:16.861288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.861427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.861427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.861427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.861716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.888008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] MFA Rules not processed for user `dc304004263343378433935147ce2105`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:16.889212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:16.950262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 793/1579] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:16.950884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.958402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 787/1580] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:16.959423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:16.962154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb5f29-2046-4bea-b3c7-3bbecaaf5604 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.967009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.967640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.967938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.968207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.968830 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.971164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44690d58-46e1-4dc1-a297-4e832af0bee9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:16.971868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:16.975941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:16.976592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:16.981778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:16.981778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:16.981778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:16.981778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.070808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 794/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:17.071639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.081777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851bdcc-72a8-402b-9f48-55defe70a8d2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.085475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 788/1582] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:16 2022] POST /identity/v3/users => generated 303 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:17.085475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.086885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.087494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.087770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.088037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.088649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.090808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.093590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b819b3-0bd4-414a-9c9b-e05255af0c70 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.098620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.099215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.099533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.099801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.100410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.101404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:17.102400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.224372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 795/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:17 2022] POST /identity/v3/users => generated 302 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:17.225280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.225999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 789/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:17 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:17.227006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.236704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11250028-8401-4ba6-91db-61df575b4373 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.237984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b951d8-0f83-4d73-9596-349ca0375c58 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.241473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.242107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.242385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.242656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.242779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.243325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.243620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.243901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.244170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.244802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.245037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.246450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.255938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 796/1585] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.256872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.260575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 790/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.260575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.268256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a04996-1a6e-4ea2-a26f-88a3a3ff1d5a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.270057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2019d4-54d2-43d1-a0a7-7e839c2a34ba admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.272829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.273417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.273695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.273963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.274581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.274752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.275376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.275657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.275934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.276567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.276682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.278135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.289640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 797/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.290273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.300800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201064d4-0966-4562-805a-810405c46a19 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.305459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.306023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.306306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.306583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.307213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.308807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.318752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 798/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.319557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.328889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d28bd-4dc2-4ef6-8ff0-2aedf832e004 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.334266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.334836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.335114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.335417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.336062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.337654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.348361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 799/1589] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.351451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.358374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5566d733-f78e-4771-83cb-5856ec2d222b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.364091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.364706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.364977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.365246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.365848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.367438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.377512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 800/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.378283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.387337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f156d5b6-61e9-4986-a084-f67981537ab8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.389440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 791/1591] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:17 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:17.390325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.446198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.447058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.447388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.447662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.448293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.449864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.452647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:17.456883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bac54d5-1b3a-4055-b0bb-37d1b97d5816 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.460458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 801/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.461329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.462051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.462840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.463178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.463562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.464271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.465929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.472055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c9994-3338-4bcd-8e91-d150dfb60fcc admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.476763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.480349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 792/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.480511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.480511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.480511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] PATH_INFO: `/v3/projects/a36ceab70491472c9201ad1ab00c627c/users/cf0393487c844bf1b142bbf101c91d3b/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.480785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.489882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e41d1a-9389-4332-8f2a-964bd5ef6ce8 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.493364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a36ceab70491472c9201ad1ab00c627c, user_id=cf0393487c844bf1b142bbf101c91d3b, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.495012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.495615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.495895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.496036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.496170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.496791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.498342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.508097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 793/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.512192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.518607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea171ba-accc-4fd6-8b5f-6d14490c497c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.523648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.524201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.524475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.524855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.525462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.526980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.540663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 794/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.540663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.547948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a12b2-8bf9-4639-b396-88e30c8be6f0 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.552529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.553088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.553363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.553634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.554241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.555792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.565974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 795/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.566726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.574768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 802/1597] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:17 2022] PUT /identity/v3/projects/a36ceab70491472c9201ad1ab00c627c/users/cf0393487c844bf1b142bbf101c91d3b/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:17.575706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.576029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8496a0a-8810-4d25-82aa-ab47a0ddb05d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.637512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc24390b-1c81-4492-bfe6-05bcf59c640c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.640565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.641803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.642081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.642350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.642843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.642968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.643471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.643751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.644024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.644582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.644693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.646847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.654518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 796/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.655374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.665307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a638d59-f097-479c-83e5-763ddbc6b647 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.669884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.672570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.672570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.672570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.672570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.673562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.683342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 797/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:17.684187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.693348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e887d7-1da7-4810-8dc4-893a190fc2ee admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.698045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.698657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.698984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.699342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] PATH_INFO: `/v3/projects/9bd235f1f9f44df4b2e3dbd0c05b3a83/users/c0aa06eea99f4011ab85ec51b0e60f2c/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.714516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd235f1f9f44df4b2e3dbd0c05b3a83, user_id=c0aa06eea99f4011ab85ec51b0e60f2c, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.717151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.739592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:17.744864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 803/1600] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:17 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:17.747810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.755240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f4303-053a-43c5-a4bc-755699e2a6f2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.759749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.760375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.760652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.760924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] PATH_INFO: `/v3/projects/7214cb5b7d4c4c949c84dc81601a89e0/users/be18a802a38143518bb0bb79afaae011/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.775568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7214cb5b7d4c4c949c84dc81601a89e0, user_id=be18a802a38143518bb0bb79afaae011, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.778223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.824217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 798/1601] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:17 2022] PUT /identity/v3/projects/9bd235f1f9f44df4b2e3dbd0c05b3a83/users/c0aa06eea99f4011ab85ec51b0e60f2c/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:17.825031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.853279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 804/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:17 2022] PUT /identity/v3/projects/7214cb5b7d4c4c949c84dc81601a89e0/users/be18a802a38143518bb0bb79afaae011/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:17.856694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:17.889811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896b149-547e-48fa-b870-f3fbb8ec51c9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.919693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:17.923708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11961332-c8ef-43e3-ae42-07d9cfed486c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:17.928567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3cea4-ec11-4317-acf6-78c03826012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.931063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3cea4-ec11-4317-acf6-78c03826012b admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.931063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3cea4-ec11-4317-acf6-78c03826012b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.931063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3cea4-ec11-4317-acf6-78c03826012b admin admin] PATH_INFO: `/v3/projects/383f186e77394f2dbb7ef0d4f1576668/users/e0232754cb924a8991a878378aed662d/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.944996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3cea4-ec11-4317-acf6-78c03826012b admin admin] RBAC: Authorizing `identity:create_grant(project_id=383f186e77394f2dbb7ef0d4f1576668, user_id=e0232754cb924a8991a878378aed662d, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.947756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d3cea4-ec11-4317-acf6-78c03826012b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:17.950793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:17.951569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:17.951845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:17.952114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] PATH_INFO: `/v3/projects/1aa84c8a680d49f8be48580c62c18889/users/fcc8eeb749a5457f98492da99199528c/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:17.967184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aa84c8a680d49f8be48580c62c18889, user_id=fcc8eeb749a5457f98492da99199528c, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:17.969869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.010401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 805/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:17 2022] PUT /identity/v3/projects/383f186e77394f2dbb7ef0d4f1576668/users/e0232754cb924a8991a878378aed662d/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:18.012361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.012695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.012967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.035575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 799/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:17 2022] PUT /identity/v3/projects/1aa84c8a680d49f8be48580c62c18889/users/fcc8eeb749a5457f98492da99199528c/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:18.036888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.042224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] MFA Rules not processed for user `cf0393487c844bf1b142bbf101c91d3b`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.049537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.099565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13067d1-20f3-46ed-b462-bb61e63c73b6 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.104264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.104917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.105195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.105468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] PATH_INFO: `/v3/projects/509dc2cd350b482c8a7ffc5abc681a78/users/ce3b48c359644912b14e05c3b7895669/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.113395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 806/1605] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.114205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.120915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=509dc2cd350b482c8a7ffc5abc681a78, user_id=ce3b48c359644912b14e05c3b7895669, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.123617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.125679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958fc1e-3f94-4411-92d1-826d0b654877 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.130446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc88521-9ac8-4df5-8db9-cd07451b86bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.131103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc88521-9ac8-4df5-8db9-cd07451b86bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.131411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc88521-9ac8-4df5-8db9-cd07451b86bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.131685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc88521-9ac8-4df5-8db9-cd07451b86bb admin admin] PATH_INFO: `/v3/projects/970f6540da024352b1c05c8a1e570ad7/users/c6b912eaa21141d18ecc9b0a7b3adbb6/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.146974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc88521-9ac8-4df5-8db9-cd07451b86bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=970f6540da024352b1c05c8a1e570ad7, user_id=c6b912eaa21141d18ecc9b0a7b3adbb6, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.149704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc88521-9ac8-4df5-8db9-cd07451b86bb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.198499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 800/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:18 2022] PUT /identity/v3/projects/509dc2cd350b482c8a7ffc5abc681a78/users/ce3b48c359644912b14e05c3b7895669/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:18.199479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.253400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 807/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:18 2022] PUT /identity/v3/projects/970f6540da024352b1c05c8a1e570ad7/users/c6b912eaa21141d18ecc9b0a7b3adbb6/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:18.254831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb03a-1e11-4920-b680-648cfb4b7e9a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.255176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb03a-1e11-4920-b680-648cfb4b7e9a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.255538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385cb03a-1e11-4920-b680-648cfb4b7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.259640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8d407-2133-4eec-bfca-22e5e7611044 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.276329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-385cb03a-1e11-4920-b680-648cfb4b7e9a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.277537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-385cb03a-1e11-4920-b680-648cfb4b7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.295607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:18.299639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:18.315985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e53e1-99ff-428b-9f85-d10fcdb4bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.316598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e53e1-99ff-428b-9f85-d10fcdb4bd6f admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.316877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e53e1-99ff-428b-9f85-d10fcdb4bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.317145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e53e1-99ff-428b-9f85-d10fcdb4bd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.317787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e53e1-99ff-428b-9f85-d10fcdb4bd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.319446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e53e1-99ff-428b-9f85-d10fcdb4bd6f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.329220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 801/1608] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:18.330538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e4240-ae7f-464d-89a4-c5121eb8ef11 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.330963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e4240-ae7f-464d-89a4-c5121eb8ef11 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.331258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e4240-ae7f-464d-89a4-c5121eb8ef11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.338008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 808/1609] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.339301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de570a6d-41ef-4b7f-b8d9-ebe93b4c9f8d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.339590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de570a6d-41ef-4b7f-b8d9-ebe93b4c9f8d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.339860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de570a6d-41ef-4b7f-b8d9-ebe93b4c9f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.354629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d88e4240-ae7f-464d-89a4-c5121eb8ef11 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.356863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d88e4240-ae7f-464d-89a4-c5121eb8ef11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.365389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 802/1610] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.366573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.366865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.367148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.370050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-de570a6d-41ef-4b7f-b8d9-ebe93b4c9f8d None None] MFA Rules not processed for user `e0232754cb924a8991a878378aed662d`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.371327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-de570a6d-41ef-4b7f-b8d9-ebe93b4c9f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.386553 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.387875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.396287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 803/1611] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.400935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.443334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 809/1612] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.444688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66582d-95c0-43ae-a596-96722cafc8b8 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.445052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66582d-95c0-43ae-a596-96722cafc8b8 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.445394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66582d-95c0-43ae-a596-96722cafc8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.454266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa32d0-aff6-4932-96a9-b196a0872fc0 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.458185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.458789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.459070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.459370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.465006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ed66582d-95c0-43ae-a596-96722cafc8b8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.466209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ed66582d-95c0-43ae-a596-96722cafc8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.474447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 810/1613] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.475653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.475934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.476215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.503646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] MFA Rules not processed for user `c6b912eaa21141d18ecc9b0a7b3adbb6`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.504905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.516967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.518417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.527281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 804/1614] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:18.528050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.538995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8fb4-63d4-47bf-9dee-712edf5f4e27 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.543598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.544360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.544647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.544918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] PATH_INFO: `/v3/projects/a685c2f726c34aaa9bc521b39735ced7/users/6a2b2d9c48e34462bfc70b36eeedb388/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.560213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a685c2f726c34aaa9bc521b39735ced7, user_id=6a2b2d9c48e34462bfc70b36eeedb388, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.563560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.573140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 811/1615] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:18.574060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.624805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:18.638855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da79e5-8438-45a1-b55b-b5068081ed93 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.643116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.645460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.645460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.645460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.654225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.655623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.665968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 812/1616] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:18.666816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.680136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed7a53-ae34-4d94-936c-87d5d4c1b841 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.683693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 805/1617] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:18 2022] PUT /identity/v3/projects/a685c2f726c34aaa9bc521b39735ced7/users/6a2b2d9c48e34462bfc70b36eeedb388/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:18.684727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.737738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.738453 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.738844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.739193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.756042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae57b6-7355-448e-9f2d-4b1e1b93cc81 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.760356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.761010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.761454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.761582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.774678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:18.811052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.815236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.824573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 813/1618] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:18.825396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.836844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9acfcd-1485-401c-a517-ed9dafc2ff0a service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.840136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.841647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.842668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.843370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.843723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.844569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.845408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.847533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.855214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 806/1619] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:18.856274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.866567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49013b50-f710-4585-8934-ef0b6de83d7b service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.870735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4803e-35dd-478b-864d-55f209d514dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.871574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4803e-35dd-478b-864d-55f209d514dd service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.871894 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4803e-35dd-478b-864d-55f209d514dd service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.872295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4803e-35dd-478b-864d-55f209d514dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.882480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4803e-35dd-478b-864d-55f209d514dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.883934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b4803e-35dd-478b-864d-55f209d514dd service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.896144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 807/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:18.897175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d677f5-502e-475c-940f-516494a4e926 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.897711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d677f5-502e-475c-940f-516494a4e926 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.897711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d677f5-502e-475c-940f-516494a4e926 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.899175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 808/1621] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 00:32:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:18.899692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d677f5-502e-475c-940f-516494a4e926 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:18.912902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d677f5-502e-475c-940f-516494a4e926 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:18.917582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fc2ae-7593-41c9-baf5-1b98d2870a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:18.918180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc2ae-7593-41c9-baf5-1b98d2870a63 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.918667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc2ae-7593-41c9-baf5-1b98d2870a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.918945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fc2ae-7593-41c9-baf5-1b98d2870a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.919850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc2ae-7593-41c9-baf5-1b98d2870a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:18.921317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fc2ae-7593-41c9-baf5-1b98d2870a63 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:18.935150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 809/1622] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:18.935150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:18.935150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:18.935150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:18.973349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:18.974532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:18.986206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 814/1623] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/projects => generated 401 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:18.986948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.070141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 810/1624] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 00:32:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:19.071256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.083105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d2988d-6385-47da-b3af-538e23e33c6f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.085501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff9910-9c75-4338-9df6-15f6914cecbc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.088771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.090079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.090360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.090626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] PATH_INFO: `/v3/projects/a685c2f726c34aaa9bc521b39735ced7/users/6a2b2d9c48e34462bfc70b36eeedb388/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.092537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e169b4-aad3-4023-8ce5-c9e4fd47f373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.093091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a685c2f726c34aaa9bc521b39735ced7, user_id=6a2b2d9c48e34462bfc70b36eeedb388, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.093320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e169b4-aad3-4023-8ce5-c9e4fd47f373 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.093643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e169b4-aad3-4023-8ce5-c9e4fd47f373 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.093917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e169b4-aad3-4023-8ce5-c9e4fd47f373 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.095034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e169b4-aad3-4023-8ce5-c9e4fd47f373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.096849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.099921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e169b4-aad3-4023-8ce5-c9e4fd47f373 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.108998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 815/1625] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:18 2022] PUT /identity/v3/projects/a685c2f726c34aaa9bc521b39735ced7/users/6a2b2d9c48e34462bfc70b36eeedb388/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:19.110155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.114559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:19.183406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd890edd-b2c8-4234-a985-e4789cbdabef admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.192740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.193347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.193624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.193893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.210081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 811/1626] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:19 2022] POST /identity/v3/users => generated 307 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:19.211916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.212717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.212717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.242178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] MFA Rules not processed for user `6a2b2d9c48e34462bfc70b36eeedb388`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:19.243799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:19.277997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:19.283732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.285755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.298208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 816/1627] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:19.299092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.334930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 812/1628] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:19.348855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.399063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9cd8b-a9c2-4677-8998-ea37150faf54 service nova] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.407002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.408194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.408843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.409337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.411057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.413390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.429542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 817/1629] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:19.434016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.447193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d2293-0a50-4c61-a1d2-ba66a82e6a4b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.453822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.454053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.454325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.454598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] PATH_INFO: `/v3/projects/6f2bf58b9fda494189085f4334d2bc47/users/43104065ca05487c894e59e92e75f5ac/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.477566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc149dd4-6bfc-44a6-b176-d0db68b4d853 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.480914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2bf58b9fda494189085f4334d2bc47, user_id=43104065ca05487c894e59e92e75f5ac, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.482153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8535cf1-fb53-4655-9559-9b16654c5976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.482852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8535cf1-fb53-4655-9559-9b16654c5976 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.483135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8535cf1-fb53-4655-9559-9b16654c5976 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.487615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8535cf1-fb53-4655-9559-9b16654c5976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.487743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.502975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8535cf1-fb53-4655-9559-9b16654c5976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.504604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8535cf1-fb53-4655-9559-9b16654c5976 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.530440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 813/1630] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:32:19.581581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e3818-438f-4fde-bf14-9242faf1db6b None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.581709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e3818-438f-4fde-bf14-9242faf1db6b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.582062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e3818-438f-4fde-bf14-9242faf1db6b None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.584115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 814/1631] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 00:32:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:32:19.595748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.596073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.605620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.605886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 818/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:19 2022] PUT /identity/v3/projects/6f2bf58b9fda494189085f4334d2bc47/users/43104065ca05487c894e59e92e75f5ac/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:19.610075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.627038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:19.628171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:19.628589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:19.726358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51c68e-2f07-46c6-b545-294b23b0f54e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.733238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.733415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.733609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.733875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.735814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.737602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.769967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 819/1633] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:19.771264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.784763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e0962-56bc-4246-8017-e2c925fcb17b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.790843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a568-ad08-490d-a92c-ec843e5f11a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:19.791611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a568-ad08-490d-a92c-ec843e5f11a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.791897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a568-ad08-490d-a92c-ec843e5f11a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.792172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a568-ad08-490d-a92c-ec843e5f11a9 admin admin] PATH_INFO: `/v3/projects/6f2bf58b9fda494189085f4334d2bc47/users/43104065ca05487c894e59e92e75f5ac/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.794157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 815/1634] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:32:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:19.803775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82a568-ad08-490d-a92c-ec843e5f11a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2bf58b9fda494189085f4334d2bc47, user_id=43104065ca05487c894e59e92e75f5ac, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:19.805429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b82a568-ad08-490d-a92c-ec843e5f11a9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:19.807607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:19.839386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 820/1635] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:19 2022] PUT /identity/v3/projects/6f2bf58b9fda494189085f4334d2bc47/users/43104065ca05487c894e59e92e75f5ac/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:19.843496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:19.844507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:19.845256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:19.904579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] MFA Rules not processed for user `43104065ca05487c894e59e92e75f5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:19.906088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:19.971363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9769d38-b181-4bdc-b8c1-5a6c40ea6394 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:19.971363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:20.008116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:20.009035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:20.009361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:20.009632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:20.043337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 821/1636] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:20.047646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:20.059032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b80c8-ca8c-4e2b-a59e-3ccb7274b19f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:20.064357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:20.065338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:20.065826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:20.066363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:20.098116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:20.099623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:20.139929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 816/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:32:20.147763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:20.151957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:20.170496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 822/1638] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:20.707187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:20.717605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26692fe4-6af4-401f-872e-8fa2576c4888 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:20.733627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d4804-f422-454d-a920-d514f3e7165a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:20.733627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d4804-f422-454d-a920-d514f3e7165a service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:20.733627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d4804-f422-454d-a920-d514f3e7165a service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:20.733627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d4804-f422-454d-a920-d514f3e7165a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:20.758399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d4804-f422-454d-a920-d514f3e7165a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:20.761440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d4804-f422-454d-a920-d514f3e7165a service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:20.787775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 817/1639] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:20 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:21.118187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:21.157630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:21.242607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742adcf7-020b-4bd8-9b10-6c8ab6484990 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:21.259563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:21.260914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:21.267865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:21.267865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] PATH_INFO: `/v3/users/e0232754cb924a8991a878378aed662d` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:21.267865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0232754cb924a8991a878378aed662d)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:21.268713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:21.363433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:21.366011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:21.366536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:21.421299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] MFA Rules not processed for user `ce3b48c359644912b14e05c3b7895669`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:21.423856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:21.533229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0232754cb924a8991a878378aed662d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:21.544985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 818/1640] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:21.550031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:21.562036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54eaf7-560d-4936-9197-3a43b32fd4cc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:21.569402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e5268-243b-4e84-930a-0977ebd479f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:21.570188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5268-243b-4e84-930a-0977ebd479f1 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:21.570464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5268-243b-4e84-930a-0977ebd479f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:21.570854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e5268-243b-4e84-930a-0977ebd479f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:21.571697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e5268-243b-4e84-930a-0977ebd479f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:21.575736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e5268-243b-4e84-930a-0977ebd479f1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:21.575891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0232754cb924a8991a878378aed662d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:21.577092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0232754cb924a8991a878378aed662d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:21.592309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0232754cb924a8991a878378aed662d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:21.659427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3548868-d5b5-4a00-97d9-27939ae912c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0232754cb924a8991a878378aed662d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:21.669738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 823/1641] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:21 2022] DELETE /identity/v3/users/e0232754cb924a8991a878378aed662d => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:21.673235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66454a85-a557-4257-b785-e0bd78b5b609 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:21.673235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66454a85-a557-4257-b785-e0bd78b5b609 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:21.673235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66454a85-a557-4257-b785-e0bd78b5b609 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:21.675019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 824/1642] 10.222.0.15 () {54 vars in 962 bytes} [Mon Aug 8 00:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:21.675673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:21.675976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:21.676249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:21.700239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:21.701492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:21.729393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 819/1643] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:21 2022] POST /identity/v3/projects => generated 405 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:21.730624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced47d4-d904-43cf-89cb-a7e6bfb02b5e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:21.732233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced47d4-d904-43cf-89cb-a7e6bfb02b5e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:21.732385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced47d4-d904-43cf-89cb-a7e6bfb02b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:21.774962 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7ced47d4-d904-43cf-89cb-a7e6bfb02b5e None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:21.777528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7ced47d4-d904-43cf-89cb-a7e6bfb02b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:21.803335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:21.848786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 825/1644] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:21.850125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:21.852106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:21.879719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc11515-eb3f-4616-825c-535d2ae5fa2b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:21.900036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 820/1645] 10.222.0.15 () {58 vars in 1059 bytes} [Mon Aug 8 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:22.176857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f30f32-3286-4951-aabc-2a0e82558665 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.176857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f30f32-3286-4951-aabc-2a0e82558665 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.176857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f30f32-3286-4951-aabc-2a0e82558665 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.176857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 821/1646] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 00:32:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:32:22.176857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.176857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.178060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.178060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:22.178060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:22.178060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 822/1647] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:32:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:22.178060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:22.178449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:22.179008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.179008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.179008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.179008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.181846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.181846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.183670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7cc71-d789-4aa6-a9cd-599bd5cbdda2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:22.183670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.183670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.183670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.184236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.184236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.184236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.186973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 823/1648] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:22.188667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:22.190569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:22.294418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7999fb8-5f54-4980-9e9d-835a85fcd1a7 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:22.304028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.305034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.305230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.305684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.331568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.342396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.352116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 826/1649] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:21 2022] POST /identity/v3/users => generated 309 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:22.353145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:22.374640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01e37b-547f-41b4-aba4-66dd3fafb544 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:22.387186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 824/1650] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:22.393239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.393872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.394157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.394522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.395141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.396714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.416788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 827/1651] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:22.421091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:22.438487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d402e0-775c-4d76-b694-61dd0356ada4 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:22.454296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.455057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.455656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.456780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] PATH_INFO: `/v3/projects/8e84df62c87e442aba44d6eba26a3f17/users/a7301ee94f9e47e2b6e4bbe0c4dfb63a/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.507521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e84df62c87e442aba44d6eba26a3f17, user_id=a7301ee94f9e47e2b6e4bbe0c4dfb63a, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.515250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.648426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 825/1652] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:22 2022] PUT /identity/v3/projects/8e84df62c87e442aba44d6eba26a3f17/users/a7301ee94f9e47e2b6e4bbe0c4dfb63a/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:22.653189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:22.669063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8d7f8-251f-447d-8538-4f0b2ee3a9e4 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:22.739441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:22.841082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.841855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.842140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.842416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.843130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.847809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.867439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:22.867703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 828/1653] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:22.886853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb120df1-b642-4d64-9ac0-24f36666d92a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:22.895910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:22.896791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:22.897175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:22.897560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] PATH_INFO: `/v3/projects/8e84df62c87e442aba44d6eba26a3f17/users/a7301ee94f9e47e2b6e4bbe0c4dfb63a/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:22.900404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e84df62c87e442aba44d6eba26a3f17, user_id=a7301ee94f9e47e2b6e4bbe0c4dfb63a, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:22.905193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:22.929294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 826/1654] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:22 2022] PUT /identity/v3/projects/8e84df62c87e442aba44d6eba26a3f17/users/a7301ee94f9e47e2b6e4bbe0c4dfb63a/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:23.924718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:23.926525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:23.971876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f0a0-488b-4691-b02f-9e5e7c10b310 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:24.217882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:24.222169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:24.223719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:24.227079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] PATH_INFO: `/v3/projects/383f186e77394f2dbb7ef0d4f1576668` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:24.233781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=383f186e77394f2dbb7ef0d4f1576668)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:24.238404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: WARNING py.warnings [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:32:24.238404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: warnings.warn(msg) Aug 08 00:32:24.238404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: Aug 08 00:32:24.243582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:24.517049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383f186e77394f2dbb7ef0d4f1576668', 'request_id': 'req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:24.524983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383f186e77394f2dbb7ef0d4f1576668', 'request_id': 'req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:24.541057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383f186e77394f2dbb7ef0d4f1576668', 'request_id': 'req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:24.633927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:24.633927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:24.636512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] The token cache is being invalidate because project 383f186e77394f2dbb7ef0d4f1576668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:24.637204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:24.640261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:24.645927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 829/1655] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:23 2022] DELETE /identity/v3/projects/383f186e77394f2dbb7ef0d4f1576668 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:24.655063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9473d4-efa9-4232-8070-1b51262f4241 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:24.672952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:24.688592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:24.692229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:24.814643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:24.815397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:24.816241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:24.816524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] PATH_INFO: `/v3/users/c6b912eaa21141d18ecc9b0a7b3adbb6` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:24.819779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b912eaa21141d18ecc9b0a7b3adbb6)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:24.821417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: WARNING py.warnings [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 00:32:24.821417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: warnings.warn(msg) Aug 08 00:32:24.821417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: Aug 08 00:32:24.825460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:24.869372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1da512-0740-4a53-b4fd-f4a54ce70d3d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:24.875364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:24.876419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:24.876912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:24.877407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] PATH_INFO: `/v3/users/43104065ca05487c894e59e92e75f5ac` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:24.878843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43104065ca05487c894e59e92e75f5ac)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:24.888631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:25.112516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b912eaa21141d18ecc9b0a7b3adbb6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.143714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43104065ca05487c894e59e92e75f5ac'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.160027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b912eaa21141d18ecc9b0a7b3adbb6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.161061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b912eaa21141d18ecc9b0a7b3adbb6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.170188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43104065ca05487c894e59e92e75f5ac'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.178975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43104065ca05487c894e59e92e75f5ac'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.199758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43104065ca05487c894e59e92e75f5ac'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.200594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b912eaa21141d18ecc9b0a7b3adbb6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.333009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5b3f7d5e-44a5-4443-87b9-63bc5370a756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43104065ca05487c894e59e92e75f5ac'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.344950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 830/1656] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:24 2022] DELETE /identity/v3/users/43104065ca05487c894e59e92e75f5ac => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:25.418590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b912eaa21141d18ecc9b0a7b3adbb6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:25.433883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 827/1657] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:24 2022] DELETE /identity/v3/users/c6b912eaa21141d18ecc9b0a7b3adbb6 => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:25.435659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:25.435958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:25.436239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:25.474756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:25.476064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:25.659492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 831/1658] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:25.853356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:25.976676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:25.980048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:26.123451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69efb99-93ad-4acf-86ff-29a4c97cc150 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:26.131330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9279f-75a9-4191-808b-3a25efe320bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:26.132044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9279f-75a9-4191-808b-3a25efe320bc service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:26.132392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9279f-75a9-4191-808b-3a25efe320bc service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:26.133250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9279f-75a9-4191-808b-3a25efe320bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:26.149754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9279f-75a9-4191-808b-3a25efe320bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:26.158403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9279f-75a9-4191-808b-3a25efe320bc service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:26.167696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 828/1659] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:27.062115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:27.064404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:27.082330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845b490-3584-48d8-92af-cf5c293bf9ec None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:27.088780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:27.090143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.090610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.091044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] PATH_INFO: `/v3/projects/6f2bf58b9fda494189085f4334d2bc47` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.092641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2bf58b9fda494189085f4334d2bc47)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:27.096118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:27.231808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2bf58b9fda494189085f4334d2bc47', 'request_id': 'req-f54e5790-4556-4210-b76f-d317108f6a22'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.241649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2bf58b9fda494189085f4334d2bc47', 'request_id': 'req-f54e5790-4556-4210-b76f-d317108f6a22'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.251452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2bf58b9fda494189085f4334d2bc47', 'request_id': 'req-f54e5790-4556-4210-b76f-d317108f6a22'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.309972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.312755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.312755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.373519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] The token cache is being invalidate because project 6f2bf58b9fda494189085f4334d2bc47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:27.375763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.375763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f54e5790-4556-4210-b76f-d317108f6a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.377932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 832/1660] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:27 2022] DELETE /identity/v3/projects/6f2bf58b9fda494189085f4334d2bc47 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:27.383637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f03834-37b4-4d76-a677-79ff5049cced None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.384153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f03834-37b4-4d76-a677-79ff5049cced None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.384720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f03834-37b4-4d76-a677-79ff5049cced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.405242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] MFA Rules not processed for user `a7301ee94f9e47e2b6e4bbe0c4dfb63a`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:27.411200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:27.446254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-94f03834-37b4-4d76-a677-79ff5049cced None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:27.447640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-94f03834-37b4-4d76-a677-79ff5049cced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:27.484703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:27.595535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 833/1661] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:27.602929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.603459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.603878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.619871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 829/1662] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:27.641937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:27.649415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:27.649559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:27.651180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:27.665522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 834/1663] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:27.676321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:27.679776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bff60-ea00-4636-b467-e67f5da2b783 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:27.686353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f136ad-48a9-4a75-ab88-963ddcd921bb None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:27.691168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:27.692585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.692866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.693146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] PATH_INFO: `/v3/projects/970f6540da024352b1c05c8a1e570ad7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.695265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=970f6540da024352b1c05c8a1e570ad7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:27.697233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:27.705096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:27.705723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.706003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.706278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.706936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:27.707262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:27.709127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:27.737767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 835/1664] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:27.746035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:27.760729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c71f7-f5fe-4b3d-9227-a39d1403496a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:27.779742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:27.780394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:27.780743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:27.781117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:27.781826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:27.783853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:27.884268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '970f6540da024352b1c05c8a1e570ad7', 'request_id': 'req-d580e996-3ca5-4207-83bb-9548d6d4d2e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.902086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '970f6540da024352b1c05c8a1e570ad7', 'request_id': 'req-d580e996-3ca5-4207-83bb-9548d6d4d2e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.920554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '970f6540da024352b1c05c8a1e570ad7', 'request_id': 'req-d580e996-3ca5-4207-83bb-9548d6d4d2e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:27.955109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 836/1665] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:27 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:27.959514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.034243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] The token cache is being invalidate because project 970f6540da024352b1c05c8a1e570ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:28.034243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:28.034243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d580e996-3ca5-4207-83bb-9548d6d4d2e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:28.039176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 830/1666] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:27 2022] DELETE /identity/v3/projects/970f6540da024352b1c05c8a1e570ad7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:28.041614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fceed3-e905-4474-92ba-5a177acc5ebe None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.041614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fceed3-e905-4474-92ba-5a177acc5ebe None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.041614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fceed3-e905-4474-92ba-5a177acc5ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.084899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f4fceed3-e905-4474-92ba-5a177acc5ebe None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:28.091102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f4fceed3-e905-4474-92ba-5a177acc5ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:28.099741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6403e9a2-4322-429d-ade3-772aeed9f1f9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.118161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 831/1667] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:28.121653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.121653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.121653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.127218 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.128226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.128719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.129201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.130038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.132227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.171631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:28.172623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:28.173795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:28.191300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 832/1668] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:28.195543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.209923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50552fda-7f64-4709-befa-5f095e803ec6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.219895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.220755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.221263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.221690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.222572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:28.223027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.224988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.243256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 837/1669] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:27 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:28.249722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.255192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 833/1670] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:28.259428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.264501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:28.291938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea5b21-9611-4d71-a901-6ca38236168c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.300018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f15f6-0b6c-4e95-b6b3-e0c19353cf4c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.311135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.313262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.313391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.313879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.313879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.314040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.314040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.315339 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.315530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.316133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.318590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.322693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.336516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 838/1671] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:28.352427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.375551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e827b2-230b-48ca-b93e-a1347959fbe8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.386796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.388439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.388921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.389393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.390192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.392429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.418382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 839/1672] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:28.424041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.440345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9195c4-175d-40a8-be4e-ed16250a3367 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.447815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784109b-3759-423e-a833-d0376be6b20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.449243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784109b-3759-423e-a833-d0376be6b20a admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.449764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784109b-3759-423e-a833-d0376be6b20a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.450292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784109b-3759-423e-a833-d0376be6b20a admin admin] PATH_INFO: `/v3/projects/caa746d83edc4ed59802ea0258afe4ed/users/8e5a59c35b024c2eaa47fdd28cfce2d2/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.469847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784109b-3759-423e-a833-d0376be6b20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa746d83edc4ed59802ea0258afe4ed, user_id=8e5a59c35b024c2eaa47fdd28cfce2d2, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.473415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784109b-3759-423e-a833-d0376be6b20a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.475023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 834/1673] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:28.479581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.569477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 840/1674] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:28 2022] PUT /identity/v3/projects/caa746d83edc4ed59802ea0258afe4ed/users/8e5a59c35b024c2eaa47fdd28cfce2d2/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:28.585330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.585947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.586490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.646166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dbc8f-f85c-46d5-be9e-81b81c7b15bb admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.678309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] MFA Rules not processed for user `8e5a59c35b024c2eaa47fdd28cfce2d2`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:28.679918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:28.795987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.796997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.797521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.798188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.798941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.801302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.805378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 841/1675] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:28.810625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.824501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b537e-d983-4769-9207-2b6b9c665293 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.830958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.831785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.832205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.833179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.833961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.836557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:28.929938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 835/1676] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/users => generated 283 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:28.933874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.955486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428758d-eeda-4368-b5e1-0d548aebdd50 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:28.961559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 842/1677] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:28.965258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:28.971061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:28.972018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:28.972366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:28.973671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:28.976749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:28.979950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.005249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 836/1678] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:29.008927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.029549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6c2ad-9dad-4af4-a140-e990ca7d4681 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.094511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c5d9c-3387-4855-b433-e9a7aa81c608 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.115400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.116052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.116481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.116757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.117499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.120365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.120834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.121613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.122173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.122628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.123842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.126176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.143084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 837/1679] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:29.144464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.161434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2682c-a5dc-41ce-b073-45aac8697cf4 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.167597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dd519-c286-4b3f-84a5-3cd9f9027082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.168635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dd519-c286-4b3f-84a5-3cd9f9027082 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.168635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dd519-c286-4b3f-84a5-3cd9f9027082 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.169042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dd519-c286-4b3f-84a5-3cd9f9027082 admin admin] PATH_INFO: `/v3/projects/0b83204fd0e649c4a01a2c015de48db0/users/4885ee9bc3c148ceb652792e2c92c267/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.196518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422dd519-c286-4b3f-84a5-3cd9f9027082 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b83204fd0e649c4a01a2c015de48db0, user_id=4885ee9bc3c148ceb652792e2c92c267, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.203878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422dd519-c286-4b3f-84a5-3cd9f9027082 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.245626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 843/1680] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:28 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:29.250955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.303263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55627cab-cce3-4801-8020-3cec164ccb28 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.321586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.322569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.339369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.344584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.344584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.350806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.352347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 838/1681] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:29 2022] PUT /identity/v3/projects/0b83204fd0e649c4a01a2c015de48db0/users/4885ee9bc3c148ceb652792e2c92c267/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:29.356717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.356717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.356717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.381156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 844/1682] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:29.382500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.401360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60cc137-9e45-4338-8d8f-a0271f7fd64e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.423687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] MFA Rules not processed for user `4885ee9bc3c148ceb652792e2c92c267`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:29.424938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:29.507738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.508424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.508702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.508976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] PATH_INFO: `/v3/projects/03d59482a27c4c85ac2400baa646750b/users/47f0b77b192d44dca9767fa88e68db42/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.532600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d59482a27c4c85ac2400baa646750b, user_id=47f0b77b192d44dca9767fa88e68db42, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.534347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.606564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 839/1683] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:29.608162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.635520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe883911-0b56-42dc-85a2-5e216c10a63a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.638524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 845/1684] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:29 2022] PUT /identity/v3/projects/03d59482a27c4c85ac2400baa646750b/users/47f0b77b192d44dca9767fa88e68db42/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:29.643860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.744214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:29.780528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.783936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.785112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.786503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.787174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.789672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.895786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc58f9-751c-4ef4-b2e9-6f741e955ffa admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:29.912576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:29.913907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:29.914329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:29.914767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:29.918754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:29.923770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:29.952777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 846/1685] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:29.972547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:29.990667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 840/1686] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:29 2022] POST /identity/v3/projects => generated 365 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:29.993184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:30.000294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3e150-217c-4d2f-9338-80920ce8a889 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:30.128948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ff069-1f29-4eff-9270-24ab2e4b0f8d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:30.144630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:30.145168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:30.145343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.145621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.145947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.146597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:30.155993 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:30.177217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c9a7a-689d-4e70-abcc-6666c721db6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:30.178096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9a7a-689d-4e70-abcc-6666c721db6c admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.178513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9a7a-689d-4e70-abcc-6666c721db6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.178932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c9a7a-689d-4e70-abcc-6666c721db6c admin admin] PATH_INFO: `/v3/projects/03d59482a27c4c85ac2400baa646750b/users/47f0b77b192d44dca9767fa88e68db42/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.186697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c9a7a-689d-4e70-abcc-6666c721db6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d59482a27c4c85ac2400baa646750b, user_id=47f0b77b192d44dca9767fa88e68db42, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:30.195324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c9a7a-689d-4e70-abcc-6666c721db6c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:30.242125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 847/1687] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:29 2022] PUT /identity/v3/projects/03d59482a27c4c85ac2400baa646750b/users/47f0b77b192d44dca9767fa88e68db42/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:30.264769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296e898-73c5-4b02-bb01-9c53c98b90dc None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.265316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296e898-73c5-4b02-bb01-9c53c98b90dc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.267267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296e898-73c5-4b02-bb01-9c53c98b90dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.274514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 841/1688] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:29 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:30.279693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:30.319557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07696a37-3985-4b8b-b30e-b633dd58ab3a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:30.381498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c296e898-73c5-4b02-bb01-9c53c98b90dc None None] MFA Rules not processed for user `47f0b77b192d44dca9767fa88e68db42`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:30.386137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c296e898-73c5-4b02-bb01-9c53c98b90dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:30.515082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 848/1689] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:30.515298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f001159-343f-4e81-8390-6b4365d23815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:30.516540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f001159-343f-4e81-8390-6b4365d23815 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.516922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f001159-343f-4e81-8390-6b4365d23815 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.517296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f001159-343f-4e81-8390-6b4365d23815 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.518320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.518612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.518892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.520275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f001159-343f-4e81-8390-6b4365d23815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:30.522022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f001159-343f-4e81-8390-6b4365d23815 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:30.550157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 842/1690] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:30.551733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee1e0e-2457-4d62-95dd-b27ba524a5ed None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.552044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee1e0e-2457-4d62-95dd-b27ba524a5ed None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.552423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee1e0e-2457-4d62-95dd-b27ba524a5ed None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.554050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 843/1691] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:30.554851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee1e0e-2457-4d62-95dd-b27ba524a5ed None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:30.594650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] MFA Rules not processed for user `fcc8eeb749a5457f98492da99199528c`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:30.595890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:30.602115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee1e0e-2457-4d62-95dd-b27ba524a5ed None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:30.622253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1977572-59db-4d78-8044-c76518a7a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:30.630817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977572-59db-4d78-8044-c76518a7a9fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.631341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977572-59db-4d78-8044-c76518a7a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.631723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1977572-59db-4d78-8044-c76518a7a9fd admin admin] PATH_INFO: `/v3/projects/14817de227fc4fd48996e93d440094dd/users/e4e4a5aee6a0477281e33bc882834dc6/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.651548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1977572-59db-4d78-8044-c76518a7a9fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=14817de227fc4fd48996e93d440094dd, user_id=e4e4a5aee6a0477281e33bc882834dc6, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:30.655020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1977572-59db-4d78-8044-c76518a7a9fd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:30.744989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 844/1692] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:30 2022] PUT /identity/v3/projects/14817de227fc4fd48996e93d440094dd/users/e4e4a5aee6a0477281e33bc882834dc6/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:30.746431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:30.746866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:30.747259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:30.773292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 849/1693] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:30.782717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:30.786131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:30.807240 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:30.811629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:30.954399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4885b45f-534a-496f-871b-fcb4c8e59994 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:30.981447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 845/1694] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:32:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:30.986689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:31.103749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc377a-a259-4bfe-81c1-a316157806cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:31.110219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc377a-a259-4bfe-81c1-a316157806cd service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.110640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc377a-a259-4bfe-81c1-a316157806cd service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.111879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc377a-a259-4bfe-81c1-a316157806cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.124432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2438f-251a-4943-a77d-9839562d16eb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:31.130334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:31.131018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.131389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.132000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.132535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:31.134309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:31.161847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 846/1695] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:31.161847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:31.164762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:31.249559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:31.311320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc377a-a259-4bfe-81c1-a316157806cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:31.323267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc377a-a259-4bfe-81c1-a316157806cd service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:31.331047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068982a9-6c23-47d8-b009-240f60187127 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:31.346372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 850/1696] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:31.346720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb4181-dac0-45c6-b321-2b5e5aab7f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:31.348443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12b0a8-cf22-4a69-89c4-ab421f2864c4 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.354057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4181-dac0-45c6-b321-2b5e5aab7f5b service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.354182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12b0a8-cf22-4a69-89c4-ab421f2864c4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.354255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4181-dac0-45c6-b321-2b5e5aab7f5b service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.354349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12b0a8-cf22-4a69-89c4-ab421f2864c4 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.354433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb4181-dac0-45c6-b321-2b5e5aab7f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.354522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 851/1697] 10.222.0.15 () {54 vars in 961 bytes} [Mon Aug 8 00:32:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:31.354522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12b0a8-cf22-4a69-89c4-ab421f2864c4 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:31.385852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12b0a8-cf22-4a69-89c4-ab421f2864c4 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:31.392881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530e43-9139-4e80-afda-cd9b5badc956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:31.395000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530e43-9139-4e80-afda-cd9b5badc956 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.395323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530e43-9139-4e80-afda-cd9b5badc956 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.395677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530e43-9139-4e80-afda-cd9b5badc956 admin admin] PATH_INFO: `/v3/projects/14817de227fc4fd48996e93d440094dd/users/e4e4a5aee6a0477281e33bc882834dc6/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.400474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530e43-9139-4e80-afda-cd9b5badc956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14817de227fc4fd48996e93d440094dd, user_id=e4e4a5aee6a0477281e33bc882834dc6, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:31.404583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530e43-9139-4e80-afda-cd9b5badc956 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:31.454192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 852/1698] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:31 2022] PUT /identity/v3/projects/14817de227fc4fd48996e93d440094dd/users/e4e4a5aee6a0477281e33bc882834dc6/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:31.455714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.459852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.460132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.551069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:31.551069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:31.569442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb4181-dac0-45c6-b321-2b5e5aab7f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:31.571094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb4181-dac0-45c6-b321-2b5e5aab7f5b service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:31.592971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 847/1699] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:31.594323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:31.596551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:31.596688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:31.737529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:31.745600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] MFA Rules not processed for user `e4e4a5aee6a0477281e33bc882834dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:31.746831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:31.928720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 853/1700] 10.222.0.15 () {58 vars in 1058 bytes} [Mon Aug 8 00:32:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:31.954243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:31.983362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113827fb-b777-4a8f-bb88-8e21dc6ab42c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:32.012501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21e96b-299b-48b2-9912-c5991fad2325 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:32.012501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e96b-299b-48b2-9912-c5991fad2325 service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:32.012501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e96b-299b-48b2-9912-c5991fad2325 service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:32.012501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21e96b-299b-48b2-9912-c5991fad2325 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:32.039091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 848/1701] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:32.074317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e96b-299b-48b2-9912-c5991fad2325 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:32.084113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21e96b-299b-48b2-9912-c5991fad2325 service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:32.139899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 854/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:32.252228 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:32.269946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:32.393924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ecc84-c911-403e-8f8f-6055b45e917e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:32.399528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fe85e-87de-4ed9-af24-5a43c7691096 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:32.399528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fe85e-87de-4ed9-af24-5a43c7691096 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:32.399528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fe85e-87de-4ed9-af24-5a43c7691096 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:32.399528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fe85e-87de-4ed9-af24-5a43c7691096 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:32.413880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fe85e-87de-4ed9-af24-5a43c7691096 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:32.415667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fe85e-87de-4ed9-af24-5a43c7691096 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:32.437036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 849/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:32.863203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19658d-0f46-4345-8d81-33e25d47d3bd None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:32.863816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19658d-0f46-4345-8d81-33e25d47d3bd None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:32.863816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19658d-0f46-4345-8d81-33e25d47d3bd None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:32.865146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 855/1704] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:32.899990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:32.900545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:32.900545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:32.936504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:32.938146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] MFA Rules not processed for user `21c1437c38d743c585ce98f7a3e96627`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:32.943198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:32.961070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 850/1705] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:32:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:32.973917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19658d-0f46-4345-8d81-33e25d47d3bd None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:32.999802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:33.000810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19658d-0f46-4345-8d81-33e25d47d3bd None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:33.001193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:33.020402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d93612-27cb-4bea-a9e3-4748c83a01d5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:33.025001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:33.028197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:33.028197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:33.028197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:33.043922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:33.044555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:33.044901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:33.045091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:33.071253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:33.072890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:33.114323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 856/1706] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:33.117197 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:33.122875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:33.207334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:33.209283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:33.263016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 851/1707] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:33.276154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9237c2-72a9-4dca-9012-b6786f03aeaa service placement] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:33.282787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:33.283826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:33.284322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:33.284788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] PATH_INFO: `/v3/users/ce3b48c359644912b14e05c3b7895669` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:33.286373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce3b48c359644912b14e05c3b7895669)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:33.288811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:33.528435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce3b48c359644912b14e05c3b7895669'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:33.561887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce3b48c359644912b14e05c3b7895669'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:33.563891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce3b48c359644912b14e05c3b7895669'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:33.577515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce3b48c359644912b14e05c3b7895669'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:33.717614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6184eaca-7bd6-4074-a614-0fdefd91f081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce3b48c359644912b14e05c3b7895669'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:33.729159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 857/1708] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:33 2022] DELETE /identity/v3/users/ce3b48c359644912b14e05c3b7895669 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:34.000238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:34.307296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:34.331954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde60083-6608-4edf-93fd-6ad1679e7def service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:34.485249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:34.486025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:34.486474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:34.488555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] PATH_INFO: `/v3/projects/509dc2cd350b482c8a7ffc5abc681a78` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:34.490952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509dc2cd350b482c8a7ffc5abc681a78)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:34.496268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:34.661210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:34.671862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:34.672486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:34.672946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:34.715011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509dc2cd350b482c8a7ffc5abc681a78', 'request_id': 'req-127e0d8e-7a87-43a7-9e75-4bcb7331e187'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:34.722889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509dc2cd350b482c8a7ffc5abc681a78', 'request_id': 'req-127e0d8e-7a87-43a7-9e75-4bcb7331e187'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:34.734556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509dc2cd350b482c8a7ffc5abc681a78', 'request_id': 'req-127e0d8e-7a87-43a7-9e75-4bcb7331e187'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:34.740260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] MFA Rules not processed for user `c0aa06eea99f4011ab85ec51b0e60f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:34.741957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:34.812298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] The token cache is being invalidate because project 509dc2cd350b482c8a7ffc5abc681a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:34.812935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:34.817150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:34.819513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 852/1709] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:34 2022] DELETE /identity/v3/projects/509dc2cd350b482c8a7ffc5abc681a78 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:34.820512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:34.822129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:34.918234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 858/1710] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:34.919488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:34.929651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:34.945053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e0d8e-7a87-43a7-9e75-4bcb7331e187 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:34.951831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:34.952707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:34.954208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:34.954795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] PATH_INFO: `/v3/users/8e5a59c35b024c2eaa47fdd28cfce2d2` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:34.956303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9550-b23c-4eab-b6ae-bfad66b7e539 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:34.960260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5a59c35b024c2eaa47fdd28cfce2d2)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:34.960260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:34.969657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:34.970310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:34.970807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:34.971281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] PATH_INFO: `/v3/users/a7301ee94f9e47e2b6e4bbe0c4dfb63a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:34.973309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7301ee94f9e47e2b6e4bbe0c4dfb63a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:34.980631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:35.227824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5a59c35b024c2eaa47fdd28cfce2d2'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.236408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7301ee94f9e47e2b6e4bbe0c4dfb63a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.253630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5a59c35b024c2eaa47fdd28cfce2d2'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.255931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5a59c35b024c2eaa47fdd28cfce2d2'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.266477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7301ee94f9e47e2b6e4bbe0c4dfb63a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.268563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7301ee94f9e47e2b6e4bbe0c4dfb63a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.269938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5a59c35b024c2eaa47fdd28cfce2d2'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.283412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7301ee94f9e47e2b6e4bbe0c4dfb63a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.396016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-dfa27787-156b-4f36-aa97-ed588cd9013a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5a59c35b024c2eaa47fdd28cfce2d2'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.405403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 853/1711] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:34 2022] DELETE /identity/v3/users/8e5a59c35b024c2eaa47fdd28cfce2d2 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:35.411849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede68a6c-869e-4f42-b0a9-b181899a36c8 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:35.411849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede68a6c-869e-4f42-b0a9-b181899a36c8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:35.411849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede68a6c-869e-4f42-b0a9-b181899a36c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:35.430364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ede68a6c-869e-4f42-b0a9-b181899a36c8 None None] MFA Rules not processed for user `c0aa06eea99f4011ab85ec51b0e60f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:35.431967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ede68a6c-869e-4f42-b0a9-b181899a36c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.508160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9f58b82-1054-475e-b8f1-6a580e1e7401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7301ee94f9e47e2b6e4bbe0c4dfb63a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:35.509822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 854/1712] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:35.511402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:35.511960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:35.512391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:35.521184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 859/1713] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:34 2022] DELETE /identity/v3/users/a7301ee94f9e47e2b6e4bbe0c4dfb63a => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:35.522925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:35.523212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:35.523531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:35.570045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:35.576660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.619137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] MFA Rules not processed for user `be18a802a38143518bb0bb79afaae011`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:35.620431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.649636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:35.714796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 860/1714] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:35.715770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:35.717100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.795028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 855/1715] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:35.796134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:35.798771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.816817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf617f-09bd-4d23-b262-ffd09425ca62 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:35.824150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:35.825260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:35.825728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:35.826003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:35.829052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.905274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:35.906283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20eab1-edee-434a-8f23-7ea877f91eb5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:35.907775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59469-d909-4076-a6fc-8501f5e2dff4 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:35.910622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:35.911353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:35.911650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:35.911912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:35.921723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 861/1716] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:35.926174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:35.926307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:35.926307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:35.926307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:35.926566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:35.934777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 856/1717] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:35.962922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] MFA Rules not processed for user `be18a802a38143518bb0bb79afaae011`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:35.964209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:35.985793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 862/1718] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:35.993562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:35.995457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:36.044782 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb07bfe-7c6d-4b1f-a17b-6e6816598d46 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:36.069871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:36.071398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:36.071398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:36.071398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:36.114340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:36.119346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:36.155579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 857/1719] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:36.182402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:36.279959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:36.282445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:36.314391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee112c02-ca6b-4bdd-9df7-911e7fa42e94 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:36.324685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:36.325659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:36.326101 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:36.326536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] PATH_INFO: `/v3/users/dc304004263343378433935147ce2105` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:36.330389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc304004263343378433935147ce2105)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:36.332680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:36.374414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:36.478070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:36.483037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:36.520404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d2a3c-a487-4d16-a445-8d8c3f255153 service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:36.527444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:36.528353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:36.528822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:36.529232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] PATH_INFO: `/v3/projects/8e84df62c87e442aba44d6eba26a3f17` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:36.530723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e84df62c87e442aba44d6eba26a3f17)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:36.533395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:36.638978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc304004263343378433935147ce2105'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.665599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc304004263343378433935147ce2105'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.672504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc304004263343378433935147ce2105'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.682535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc304004263343378433935147ce2105'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.766803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e84df62c87e442aba44d6eba26a3f17', 'request_id': 'req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.778596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a935e081-5059-43b7-96bf-e122b78e5b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc304004263343378433935147ce2105'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.788827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e84df62c87e442aba44d6eba26a3f17', 'request_id': 'req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.792994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 863/1720] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:36 2022] DELETE /identity/v3/users/dc304004263343378433935147ce2105 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:36.795209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612e32e-e6fc-4112-9946-d2fae50fa0be None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:36.795770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612e32e-e6fc-4112-9946-d2fae50fa0be None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:36.796190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612e32e-e6fc-4112-9946-d2fae50fa0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:36.815456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e84df62c87e442aba44d6eba26a3f17', 'request_id': 'req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.823203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5612e32e-e6fc-4112-9946-d2fae50fa0be None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:36.825986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5612e32e-e6fc-4112-9946-d2fae50fa0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:36.902308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] The token cache is being invalidate because project 8e84df62c87e442aba44d6eba26a3f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:36.902755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.903565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fb54c63f-b2f8-4fcc-995e-fe105c4e59f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:36.917268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 858/1721] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:36 2022] DELETE /identity/v3/projects/8e84df62c87e442aba44d6eba26a3f17 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:36.935961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:36.936387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:36.936806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:36.985872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:36.987960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.026244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 864/1722] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:37.028910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 859/1723] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:37.043933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf573a-2b5f-4fea-8e7b-a741bdd8ab1d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.044485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf573a-2b5f-4fea-8e7b-a741bdd8ab1d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.045041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf573a-2b5f-4fea-8e7b-a741bdd8ab1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.062697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.064897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.072144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-22cf573a-2b5f-4fea-8e7b-a741bdd8ab1d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:37.100100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-22cf573a-2b5f-4fea-8e7b-a741bdd8ab1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.100100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 865/1724] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:37.100100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f39b0-0573-43f8-a74e-f9f16f4342e2 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.100100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f39b0-0573-43f8-a74e-f9f16f4342e2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.100100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f39b0-0573-43f8-a74e-f9f16f4342e2 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.100100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 866/1725] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 00:32:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:37.100721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f39b0-0573-43f8-a74e-f9f16f4342e2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.142939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f39b0-0573-43f8-a74e-f9f16f4342e2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:37.151561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:37.151561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.151561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.151561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.152451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:37.152451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:37.152734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c7866-46a8-4f28-b9ac-9b2f3238e3b9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:37.172180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 867/1726] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:37.173504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.173792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.174065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.239075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf58de4-7044-4dbf-8612-15e11eb364d4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:37.260289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:37.262706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.262995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.265402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.284342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:37.286198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.294654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:37.296169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:37.313111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 868/1727] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:32:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:37.316527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.320240 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 860/1728] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:37.321256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.322722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.336385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a41966-5cfd-4b11-88bf-c0ab770db9d9 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:37.341749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1370f6-8737-443b-ab23-d14270b93945 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:37.347461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:37.348292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.348677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.349109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] PATH_INFO: `/v3/users/cf0393487c844bf1b142bbf101c91d3b` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.349928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:37.350626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.351192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0393487c844bf1b142bbf101c91d3b)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:37.351630 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.352017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.352842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:37.353733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:37.356505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:37.514071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 869/1729] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:37 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:37.515910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.520700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.590921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0393487c844bf1b142bbf101c91d3b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:37.625073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0393487c844bf1b142bbf101c91d3b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:37.628689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0393487c844bf1b142bbf101c91d3b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:37.660730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0393487c844bf1b142bbf101c91d3b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:37.687991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:37.779590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca92202-d92d-4806-9d5e-59b30e3fd73d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:37.791392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0393487c844bf1b142bbf101c91d3b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:37.797618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 861/1730] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:37 2022] DELETE /identity/v3/users/cf0393487c844bf1b142bbf101c91d3b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:37.798495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.799920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.804832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:37.805562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.806692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.806967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.811707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.828929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:37.830620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103cef6-cf0b-45d6-8127-54fd5a03b49c service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:37.851460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 870/1731] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:37.852937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408a84-e1de-4080-a552-7676a1ec05d2 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:37.853227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408a84-e1de-4080-a552-7676a1ec05d2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:37.853508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408a84-e1de-4080-a552-7676a1ec05d2 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:37.860364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 871/1732] 10.222.0.15 () {54 vars in 912 bytes} [Mon Aug 8 00:32:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:37.861424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408a84-e1de-4080-a552-7676a1ec05d2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:37.865966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5a408a84-e1de-4080-a552-7676a1ec05d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:37.978098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67855e15-03ca-40a5-96d6-0b44e05e2caf admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:38.002725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:38.004341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.006305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.006748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] PATH_INFO: `/v3/projects/caa746d83edc4ed59802ea0258afe4ed` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.012549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa746d83edc4ed59802ea0258afe4ed)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:38.015089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:38.070394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408a84-e1de-4080-a552-7676a1ec05d2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:38.078898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac196be7-00a4-42a5-822c-09b28e57a1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:38.086121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196be7-00a4-42a5-822c-09b28e57a1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.086121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196be7-00a4-42a5-822c-09b28e57a1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.086121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac196be7-00a4-42a5-822c-09b28e57a1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.086121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac196be7-00a4-42a5-822c-09b28e57a1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:38.086121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac196be7-00a4-42a5-822c-09b28e57a1c9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:38.177512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa746d83edc4ed59802ea0258afe4ed', 'request_id': 'req-65705bcd-495e-4a47-bb88-001e5675d09f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.184897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa746d83edc4ed59802ea0258afe4ed', 'request_id': 'req-65705bcd-495e-4a47-bb88-001e5675d09f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.193266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 872/1733] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:37 2022] POST /identity/v3/users => generated 287 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:38.194883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b38c4f-0f3b-4842-bf17-a6612177cb8e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.195178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b38c4f-0f3b-4842-bf17-a6612177cb8e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.195656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b38c4f-0f3b-4842-bf17-a6612177cb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.198521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa746d83edc4ed59802ea0258afe4ed', 'request_id': 'req-65705bcd-495e-4a47-bb88-001e5675d09f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.224259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b8b38c4f-0f3b-4842-bf17-a6612177cb8e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:38.225351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b8b38c4f-0f3b-4842-bf17-a6612177cb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.260063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:38.310108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] The token cache is being invalidate because project caa746d83edc4ed59802ea0258afe4ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:38.310108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.310108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-65705bcd-495e-4a47-bb88-001e5675d09f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.310108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 862/1734] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:37 2022] DELETE /identity/v3/projects/caa746d83edc4ed59802ea0258afe4ed => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:38.310726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.310804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.311569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.341283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:38.341815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 873/1735] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:38.342823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.347247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.347565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.347843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.368879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] MFA Rules not processed for user `fcc8eeb749a5457f98492da99199528c`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:38.370280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.409671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 863/1736] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:32:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:38.410677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:38.413703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.452398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204573a-76c1-4f3b-a641-59bb5e0e4ed9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:38.463582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:38.467251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.467251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.467251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] PATH_INFO: `/v3/users/4885ee9bc3c148ceb652792e2c92c267` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.467876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4885ee9bc3c148ceb652792e2c92c267)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:38.470291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:38.486092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 874/1737] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:38.487110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:38.488666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.504399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6a4f3-7f8d-4580-b12c-384ecc224868 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:38.510793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:38.512818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.512818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.512818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.512818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:38.517925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:38.531539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 875/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:38.532595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:38.534338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.552368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa046c-c42d-4913-959b-865a9e291174 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:38.563562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:38.563562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.563562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.563562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] PATH_INFO: `/v3/projects/efcecb80f0c64612a23a742d93b36380` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.563950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efcecb80f0c64612a23a742d93b36380)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:38.565185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:38.693991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4885ee9bc3c148ceb652792e2c92c267'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.712001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4885ee9bc3c148ceb652792e2c92c267'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.713133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4885ee9bc3c148ceb652792e2c92c267'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.731904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4885ee9bc3c148ceb652792e2c92c267'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.743369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efcecb80f0c64612a23a742d93b36380', 'request_id': 'req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.750060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efcecb80f0c64612a23a742d93b36380', 'request_id': 'req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.757834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efcecb80f0c64612a23a742d93b36380', 'request_id': 'req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.842464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4885ee9bc3c148ceb652792e2c92c267'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.856266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 864/1739] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:38 2022] DELETE /identity/v3/users/4885ee9bc3c148ceb652792e2c92c267 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:38.857583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:38.863481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.921975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] The token cache is being invalidate because project efcecb80f0c64612a23a742d93b36380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:38.924746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.924746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:38.925766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 876/1740] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:38 2022] DELETE /identity/v3/projects/efcecb80f0c64612a23a742d93b36380 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:38.926673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:38.932223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:38.955041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe8e3b-1c13-4a08-bfea-ba5a332292f2 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:38.960770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:38.961562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:38.961697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:38.961985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] PATH_INFO: `/v3/users/47f0b77b192d44dca9767fa88e68db42` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:38.963269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f0b77b192d44dca9767fa88e68db42)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:38.965176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.021856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2165a1-9ea3-4d54-8a8c-53d666fd7bb5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:39.073953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:39.074463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.074739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.075014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.076949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.089521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:39.090914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.102532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 877/1741] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:39.103415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:39.114469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5053e5e-0b31-4aea-a24d-3a62461e5817 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:39.120598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:39.121291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.121577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.121857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.124285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.137327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:39.138781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.147589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 878/1742] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:39.148488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:39.149921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.168140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f82a06-4ccb-4dee-9db2-c351ddd41326 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:39.174097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:39.174694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.175046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.175435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.176184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:39.178049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.191413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 879/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:39.191413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:39.195295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.204563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f0b77b192d44dca9767fa88e68db42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.213468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:39.227413 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f0b77b192d44dca9767fa88e68db42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.228519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f0b77b192d44dca9767fa88e68db42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.243292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f0b77b192d44dca9767fa88e68db42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.297875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27cc0b-626a-4a60-b36f-96cb18227b74 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:39.305837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:39.306589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.306973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.307398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.309503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.321981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:39.323683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dffce1-65f6-4b56-857e-3f5e60a6f14a service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.338574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 880/1744] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:39.339142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878234c-8962-4645-b188-ef0782942006 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.339515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878234c-8962-4645-b188-ef0782942006 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.339855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8878234c-8962-4645-b188-ef0782942006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.361929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8878234c-8962-4645-b188-ef0782942006 None None] MFA Rules not processed for user `c0aa06eea99f4011ab85ec51b0e60f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:39.363515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8878234c-8962-4645-b188-ef0782942006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.371943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b17eee59-a011-44a8-9f4d-f9715b32f573 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f0b77b192d44dca9767fa88e68db42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.371943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 865/1745] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:38 2022] DELETE /identity/v3/users/47f0b77b192d44dca9767fa88e68db42 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:39.371943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.372202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.372309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.415379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:39.419150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.516146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 881/1746] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:39.517074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878234c-8962-4645-b188-ef0782942006 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:39.520771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8878234c-8962-4645-b188-ef0782942006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.544206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 866/1747] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:39.545576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:39.551685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.554595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8878234c-8962-4645-b188-ef0782942006 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:39.565005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:39.566619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.567335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.572277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] PATH_INFO: `/v3/users/6a2b2d9c48e34462bfc70b36eeedb388` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.572277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a2b2d9c48e34462bfc70b36eeedb388)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:39.573787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.586220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c73908-e25c-4a63-a20b-5b436435f696 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:39.599981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:39.600839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:39.601449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:39.601874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] PATH_INFO: `/v3/projects/1d370ba54d034a819ac152cadfbdb636/users/26b3d455a51d4e1d85aac4903d5f72d0/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:39.631754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d370ba54d034a819ac152cadfbdb636, user_id=26b3d455a51d4e1d85aac4903d5f72d0, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:39.638410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:39.740806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 867/1748] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:39 2022] PUT /identity/v3/projects/1d370ba54d034a819ac152cadfbdb636/users/26b3d455a51d4e1d85aac4903d5f72d0/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:39.743086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:39.748786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:39.878919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a2b2d9c48e34462bfc70b36eeedb388'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.923080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a2b2d9c48e34462bfc70b36eeedb388'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.930514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a2b2d9c48e34462bfc70b36eeedb388'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.967933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a2b2d9c48e34462bfc70b36eeedb388'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:39.992456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb1c49-b99f-4b7b-be4e-d3d94184134a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:40.018774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:40.023322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:40.027847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:40.027847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:40.081544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9d87496e-19fb-4905-9177-0f1c83ac7027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a2b2d9c48e34462bfc70b36eeedb388'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:40.091113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 882/1749] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:39 2022] DELETE /identity/v3/users/6a2b2d9c48e34462bfc70b36eeedb388 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:40.091998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:40.092377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:40.093078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:40.201969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:40.206979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] MFA Rules not processed for user `26b3d455a51d4e1d85aac4903d5f72d0`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:40.210764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:40.279555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:40.281657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:40.323851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 868/1750] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:40.369134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 883/1751] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:40.373251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:40.379885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:40.383430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:40.393520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d7652-0428-4e72-b6cd-205e3402da1a service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:40.551140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:40.553734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:40.555052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:40.557093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:40.558648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:40.564001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:40.570877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c63d00-2f33-4321-a4cf-42dc489e40ea None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:40.600220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:40.603534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:40.603534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:40.603534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] PATH_INFO: `/v3/projects/a36ceab70491472c9201ad1ab00c627c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:40.603966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a36ceab70491472c9201ad1ab00c627c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:40.605921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:40.676304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 869/1752] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:40 2022] POST /identity/v3/projects => generated 371 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:40.681398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:40.689418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:40.866638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a36ceab70491472c9201ad1ab00c627c', 'request_id': 'req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:40.896687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a36ceab70491472c9201ad1ab00c627c', 'request_id': 'req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:40.927323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e394e30-555e-4304-a51a-cec2ab4c1a57 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:40.931513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a36ceab70491472c9201ad1ab00c627c', 'request_id': 'req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.056646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] The token cache is being invalidate because project a36ceab70491472c9201ad1ab00c627c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:41.056646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.056822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1afe1ca3-d78a-4d09-bc9a-7940a46f2312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.063979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 884/1753] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:40 2022] DELETE /identity/v3/projects/a36ceab70491472c9201ad1ab00c627c => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:41.070723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.070879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.071845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.151098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:41.155667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.201530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5931b-1d08-431e-a8f7-2a238e5a644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:41.202462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5931b-1d08-431e-a8f7-2a238e5a644d admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.204021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5931b-1d08-431e-a8f7-2a238e5a644d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.205248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5931b-1d08-431e-a8f7-2a238e5a644d admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.207421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5931b-1d08-431e-a8f7-2a238e5a644d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:41.212245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5931b-1d08-431e-a8f7-2a238e5a644d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:41.291455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 885/1754] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:41.292980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:41.297657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.314789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d0651-fb32-4b87-a541-d203c958af71 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:41.321277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:41.322205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.322391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.323110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] PATH_INFO: `/v3/projects/03d59482a27c4c85ac2400baa646750b` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.324529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d59482a27c4c85ac2400baa646750b)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:41.330660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:41.420268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 870/1755] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:40 2022] POST /identity/v3/users => generated 292 bytes in 739 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:41.421919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.422540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.422911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.461160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:41.462371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.478577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 871/1756] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:41.485226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:41.485226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.492364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d59482a27c4c85ac2400baa646750b', 'request_id': 'req-76725ae5-a442-4176-8095-ca40201f7c7a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.492680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40b8b8-171c-47f9-97d0-5ef1692d88b5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:41.498269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d59482a27c4c85ac2400baa646750b', 'request_id': 'req-76725ae5-a442-4176-8095-ca40201f7c7a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.521135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d59482a27c4c85ac2400baa646750b', 'request_id': 'req-76725ae5-a442-4176-8095-ca40201f7c7a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.572594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:41.572594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.572594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.572594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.573109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:41.573268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:41.591475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 872/1757] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:41.591475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:41.593162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.595063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:41.604069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] The token cache is being invalidate because project 03d59482a27c4c85ac2400baa646750b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:41.604441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.606099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.609683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 886/1758] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:41 2022] DELETE /identity/v3/projects/03d59482a27c4c85ac2400baa646750b => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:41.611049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:41.619246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.635708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76725ae5-a442-4176-8095-ca40201f7c7a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:41.637533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6e6a5-7614-42da-b1d3-cc83d2c792af admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:41.642913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:41.655114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.655326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.655656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.656801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:41.657101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:41.657400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:41.657884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.658215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.658544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] PATH_INFO: `/v3/projects/0b83204fd0e649c4a01a2c015de48db0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.659008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:41.661277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b83204fd0e649c4a01a2c015de48db0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:41.667211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:41.699753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 887/1759] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:41.702537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:41.719003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:41.733003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f00bb9-1dd1-43f6-b682-0b29916b84ee admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:41.738867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:41.739781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:41.740266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:41.740858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] PATH_INFO: `/v3/projects/a685c2f726c34aaa9bc521b39735ced7` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:41.742247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a685c2f726c34aaa9bc521b39735ced7)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:41.758292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:41.837251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b83204fd0e649c4a01a2c015de48db0', 'request_id': 'req-a101217e-eacb-454c-aea7-a5d126c8048c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.842919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b83204fd0e649c4a01a2c015de48db0', 'request_id': 'req-a101217e-eacb-454c-aea7-a5d126c8048c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.851444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b83204fd0e649c4a01a2c015de48db0', 'request_id': 'req-a101217e-eacb-454c-aea7-a5d126c8048c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.921823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a685c2f726c34aaa9bc521b39735ced7', 'request_id': 'req-77129fc3-43cd-4ae8-8452-28b32958ad0a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.927413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a685c2f726c34aaa9bc521b39735ced7', 'request_id': 'req-77129fc3-43cd-4ae8-8452-28b32958ad0a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.934173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a685c2f726c34aaa9bc521b39735ced7', 'request_id': 'req-77129fc3-43cd-4ae8-8452-28b32958ad0a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.973615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] The token cache is being invalidate because project 0b83204fd0e649c4a01a2c015de48db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:41.974158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.975094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:41.976699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 873/1760] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:41 2022] DELETE /identity/v3/projects/0b83204fd0e649c4a01a2c015de48db0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:41.980687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:41.982246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.100294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101217e-eacb-454c-aea7-a5d126c8048c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:42.108163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] The token cache is being invalidate because project a685c2f726c34aaa9bc521b39735ced7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:42.108754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.110225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-77129fc3-43cd-4ae8-8452-28b32958ad0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.112624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 888/1761] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:41 2022] DELETE /identity/v3/projects/a685c2f726c34aaa9bc521b39735ced7 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:42.153227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.153227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.153227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.154620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:42.156592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.213633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:42.214314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.214619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.214856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] PATH_INFO: `/v3/projects/1227885f866b4b98b38ebe72008428b1/users/4c469e758c8d48e5950ca7632be0287b/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.244611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1227885f866b4b98b38ebe72008428b1, user_id=4c469e758c8d48e5950ca7632be0287b, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:42.247471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:42.252641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 889/1762] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:42.253512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:42.254991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.272134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713a10d-f202-41c1-9499-e897f779aed2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:42.279323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c81d0-b6fb-4873-87b0-ff0e70aa6790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:42.280618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c81d0-b6fb-4873-87b0-ff0e70aa6790 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.281178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c81d0-b6fb-4873-87b0-ff0e70aa6790 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.281622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c81d0-b6fb-4873-87b0-ff0e70aa6790 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.282441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c81d0-b6fb-4873-87b0-ff0e70aa6790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:42.284806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c81d0-b6fb-4873-87b0-ff0e70aa6790 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:42.358397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 874/1763] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:41 2022] PUT /identity/v3/projects/1227885f866b4b98b38ebe72008428b1/users/4c469e758c8d48e5950ca7632be0287b/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:42.360848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:42.364711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.409770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 890/1764] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:42 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:42.410300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb437-d576-4b42-af46-d4e25c08c4e5 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.410593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb437-d576-4b42-af46-d4e25c08c4e5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.410914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bb437-d576-4b42-af46-d4e25c08c4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.440239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64e74a-2647-40db-8f37-05a9bde7c523 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:42.447103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-bb1bb437-d576-4b42-af46-d4e25c08c4e5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:42.448495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bb1bb437-d576-4b42-af46-d4e25c08c4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.548392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:42.549615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.550055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.550552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] PATH_INFO: `/v3/users/e4e4a5aee6a0477281e33bc882834dc6` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.551969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4e4a5aee6a0477281e33bc882834dc6)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:42.554593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:42.589960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 891/1765] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:42.594377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.594377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.594377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.619315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:42.622968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.629252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 892/1766] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:42.630090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:42.631656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:42.643428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a0fea-5e83-47a7-b961-f7fad948598d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:42.657515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:42.657515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.657515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.657515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.657995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:42.657995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:42.670575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 893/1767] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:42.670575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:42.692571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8dbf8-5937-4f9c-97f8-0c2412d95bac admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:42.697258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:42.701485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.701485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.701485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.701485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:42.701485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:42.721519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:42.811161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4e4a5aee6a0477281e33bc882834dc6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.826399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4e4a5aee6a0477281e33bc882834dc6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.828443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4e4a5aee6a0477281e33bc882834dc6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.836238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4e4a5aee6a0477281e33bc882834dc6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.972995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-304915f3-7baa-4df5-995c-9bffee09ea11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4e4a5aee6a0477281e33bc882834dc6'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:42.979842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 875/1768] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:42 2022] DELETE /identity/v3/users/e4e4a5aee6a0477281e33bc882834dc6 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:42.984524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:42.984524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:42.984524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:42.985664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 894/1769] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:42 2022] POST /identity/v3/users => generated 288 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:42.987105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:42.989009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.002670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1a510-4894-4fb3-bfc5-5724929508f9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.020218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:43.021446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.090148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.090916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.095394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.095773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.096495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:43.096803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.098549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.112965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 876/1770] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:43.114242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.116463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.127107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 895/1771] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:43.128071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.135829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f0b576-8756-4b2e-b08a-b544abc6b040 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.136219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.143519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.143519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.143519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.143519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] PATH_INFO: `/v3/projects/1227885f866b4b98b38ebe72008428b1/users/4c469e758c8d48e5950ca7632be0287b/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.145249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1227885f866b4b98b38ebe72008428b1, user_id=4c469e758c8d48e5950ca7632be0287b, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.148150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.156182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4b7a4-487c-4f53-8307-8050dcc14dd8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.179741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 877/1772] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:43 2022] PUT /identity/v3/projects/1227885f866b4b98b38ebe72008428b1/users/4c469e758c8d48e5950ca7632be0287b/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:43.179741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.237952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.238473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.239059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.239059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.239688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.241291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.251917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 896/1773] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:43.252833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.266241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9b7ba-c304-490c-8dc6-32f96a7608f5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.266670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726f8ff-a41e-48c1-9632-aec59bb27d07 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.271370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.271703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.272358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.272583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.272689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.272844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.272949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.273120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.273565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:43.273815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.273898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.275454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b8aa8-e4f6-40cc-b08a-695fb1b387f9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.275734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.300103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 878/1774] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:43.301630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.301929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.302166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.337336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] MFA Rules not processed for user `4c469e758c8d48e5950ca7632be0287b`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:43.338607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.376280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 897/1775] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:43 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:43.377577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.447130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca99c7b-6371-4b21-9234-4e9b92cd2335 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.451356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 879/1776] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:43.451356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.458309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.459179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.459660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.460129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.460934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.462766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.467176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea825b-cb52-4661-beaa-e62f749e0329 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.472526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.473233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.473578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.473919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.474663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.479209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 898/1777] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:43.479209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.479414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.486867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ce031-8640-412c-a287-7d03cc5f2610 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.491855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.492369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.492652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.492974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.493547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.495502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.596935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 880/1778] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:43 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:43.598350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.601384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.621676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 899/1779] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:43 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:43.623024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.634499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29d8ca-027b-4afa-a3aa-dce588459818 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.681605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d8ef8-e235-49a5-b34f-23f4eaad1610 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.698635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.699591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.700120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.700560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.718487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.719266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.719735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.720080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] PATH_INFO: `/v3/projects/65fc9fdf93124d50b1c783a9271f1faf/users/ed408d27ba5b46978e5afa4d4012d1c1/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.739471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fc9fdf93124d50b1c783a9271f1faf, user_id=ed408d27ba5b46978e5afa4d4012d1c1, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.742582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.784720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.786300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.803688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 881/1780] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:43.804569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.805884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:43.821102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de0961-ee4c-4924-bf83-cce5d8989c46 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.825824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.826575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.826920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.827275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] PATH_INFO: `/v3/projects/14817de227fc4fd48996e93d440094dd` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.828451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14817de227fc4fd48996e93d440094dd)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.830355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.842123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 900/1781] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:43 2022] PUT /identity/v3/projects/65fc9fdf93124d50b1c783a9271f1faf/users/ed408d27ba5b46978e5afa4d4012d1c1/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:43.843010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:43.849942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:43.925029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3ffd-241d-492e-a8d0-61d0df4cc95e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:43.940599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:43.942447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:43.943365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:43.943802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:43.944819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:43.951086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:43.974693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14817de227fc4fd48996e93d440094dd', 'request_id': 'req-b58e8972-3988-4201-bb1b-9222663f00a3'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:43.981505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14817de227fc4fd48996e93d440094dd', 'request_id': 'req-b58e8972-3988-4201-bb1b-9222663f00a3'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:43.988508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14817de227fc4fd48996e93d440094dd', 'request_id': 'req-b58e8972-3988-4201-bb1b-9222663f00a3'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:44.082916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 901/1782] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:43 2022] POST /identity/v3/users => generated 284 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:44.084179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.096773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a27d1f-b59d-41f2-99ea-081756562daf admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.116394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] The token cache is being invalidate because project 14817de227fc4fd48996e93d440094dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:44.117295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:44.118380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b58e8972-3988-4201-bb1b-9222663f00a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:44.121025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 882/1783] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:43 2022] DELETE /identity/v3/projects/14817de227fc4fd48996e93d440094dd => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:44.122966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.123452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.123882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.164311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] MFA Rules not processed for user `ed408d27ba5b46978e5afa4d4012d1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:44.165986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.183415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.184085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.184403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.184994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.185887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.191975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.208673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 902/1784] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:44.209967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.212540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.267284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 883/1785] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:44.268308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.269767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.293002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2f95a-4b8a-401c-af9b-f1eac3ead974 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.301412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae35e3-095c-4137-8f3c-94aa265bc1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.301412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae35e3-095c-4137-8f3c-94aa265bc1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.301412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae35e3-095c-4137-8f3c-94aa265bc1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.301412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae35e3-095c-4137-8f3c-94aa265bc1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.301905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae35e3-095c-4137-8f3c-94aa265bc1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.301995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811bf64-c4e0-4292-aa0b-c45775ca49ac admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.302102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae35e3-095c-4137-8f3c-94aa265bc1d0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.305045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.305752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.306160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.306501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.308470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.316744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 884/1786] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:44.316744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.316744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.316744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.346494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:44.348095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.370459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 885/1787] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:44.371497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.374020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.396657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e69c2-456b-458a-92a2-aed56f894ad7 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.399061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.399751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.400060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.400342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.400996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.402724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.416681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 886/1788] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:44.419751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.424677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.426891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.440723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.456544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 903/1789] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:44.457795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.507331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8a620-c013-4af0-bbc5-7842bd4a6b6d service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.526743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c0891-d89e-487b-89da-e7362a26b580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.527401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c0891-d89e-487b-89da-e7362a26b580 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.527679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c0891-d89e-487b-89da-e7362a26b580 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.527974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c0891-d89e-487b-89da-e7362a26b580 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.529083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c0891-d89e-487b-89da-e7362a26b580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.533202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c0891-d89e-487b-89da-e7362a26b580 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.553145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 904/1790] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:44.556589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bb5e5-f8a4-4f69-90ae-77f659e7688e admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.560253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.560561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.560837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.569750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.572756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.572756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.572756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.597253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:44.603472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.604898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:44.605123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.611135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:44.622365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 905/1791] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:44.623411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.633794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 887/1792] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:44.634771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.648343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc333-fe81-4af7-a4c7-10b54b86b5f7 service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.661131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.661131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.661131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.661131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] PATH_INFO: `/v3/projects/cc1ca37a67064d1f9a40297f7c5722ff/users/909e20a8b2a24f759dfe9fe33dcbf56f/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.662276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32425c99-a06b-4c0c-bee3-c735f2d0ca7b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:44.692168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bd15a-6e6e-4bfc-93c3-a88e120f85d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:44.692168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bd15a-6e6e-4bfc-93c3-a88e120f85d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.692168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bd15a-6e6e-4bfc-93c3-a88e120f85d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.692168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bd15a-6e6e-4bfc-93c3-a88e120f85d6 admin admin] PATH_INFO: `/v3/projects/8191bccb93da4bf5ad496f82641cfeae/users/60136c1889d0478b86ab68b4131b0b97/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.695002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1ca37a67064d1f9a40297f7c5722ff, user_id=909e20a8b2a24f759dfe9fe33dcbf56f, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.698834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.743427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bd15a-6e6e-4bfc-93c3-a88e120f85d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8191bccb93da4bf5ad496f82641cfeae, user_id=60136c1889d0478b86ab68b4131b0b97, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:44.751020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bd15a-6e6e-4bfc-93c3-a88e120f85d6 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:44.863602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 888/1793] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:44 2022] PUT /identity/v3/projects/cc1ca37a67064d1f9a40297f7c5722ff/users/909e20a8b2a24f759dfe9fe33dcbf56f/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:44.863602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:44.878686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 906/1794] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:44 2022] PUT /identity/v3/projects/8191bccb93da4bf5ad496f82641cfeae/users/60136c1889d0478b86ab68b4131b0b97/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:44.880410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21af8-511d-4b07-ae1e-3ab7b12bc94f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:44.880801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21af8-511d-4b07-ae1e-3ab7b12bc94f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:44.881182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21af8-511d-4b07-ae1e-3ab7b12bc94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:44.889582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:44.957151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-edd21af8-511d-4b07-ae1e-3ab7b12bc94f None None] MFA Rules not processed for user `909e20a8b2a24f759dfe9fe33dcbf56f`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:44.965370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-edd21af8-511d-4b07-ae1e-3ab7b12bc94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:45.108670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7496697-f84d-4e11-aaaf-a28572d57644 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:45.199303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 907/1795] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:45.203034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.203789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.204381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.308264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] MFA Rules not processed for user `60136c1889d0478b86ab68b4131b0b97`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:45.310343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:45.326650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:45.328673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.329325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.329872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.331174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:45.333699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:45.336924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:45.361783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 889/1796] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:45.362877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:45.392147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2849cf4d-5d13-4af7-b86f-9294462ee5e7 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:45.405515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:45.407674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.408229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.408689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.409535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:45.411829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:45.458291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 908/1797] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:45.459249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:45.476663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60d2c2-2a85-44b6-b2c2-d4938584f773 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:45.483343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:45.484080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.484435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.484707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.485442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:45.487501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:45.553243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 890/1798] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:45 2022] POST /identity/v3/projects => generated 369 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:45.554262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:45.555970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:45.581207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 909/1799] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:45 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:45.581207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:45.649143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf97f-780e-4171-baea-4c96ad48ee7f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:45.670003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfd7f48-bbe2-4f95-858e-4fb8f5264bfa admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:45.679025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b11531-a6c7-441e-88cd-93b8342b1d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:45.680007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b11531-a6c7-441e-88cd-93b8342b1d12 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.680455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b11531-a6c7-441e-88cd-93b8342b1d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.680866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b11531-a6c7-441e-88cd-93b8342b1d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.681649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b11531-a6c7-441e-88cd-93b8342b1d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:45.683847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b11531-a6c7-441e-88cd-93b8342b1d12 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:45.730486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:45.731296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.731586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.731860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] PATH_INFO: `/v3/users/ed408d27ba5b46978e5afa4d4012d1c1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.733532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed408d27ba5b46978e5afa4d4012d1c1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:45.737650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:45.818797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 910/1800] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:45 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:45.818797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:45.818797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:45.818797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:45.877641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:45.879178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:45.982383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 911/1801] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:32:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:45.983363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.059533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43126d-5063-48f1-9523-df54c89b8867 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.065708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.066395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.066677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.066950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.067732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.069832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.085885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed408d27ba5b46978e5afa4d4012d1c1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:46.100740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed408d27ba5b46978e5afa4d4012d1c1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:46.102727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed408d27ba5b46978e5afa4d4012d1c1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:46.116825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed408d27ba5b46978e5afa4d4012d1c1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:46.262265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed408d27ba5b46978e5afa4d4012d1c1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:46.271059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 912/1803] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:45 2022] POST /identity/v3/users => generated 291 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:46.272806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.274703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 891/1803] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:45 2022] DELETE /identity/v3/users/ed408d27ba5b46978e5afa4d4012d1c1 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:46.275722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.276705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.277200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.290745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902362c5-6a20-41e9-a64b-71dd5def8c2c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.392322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81014a-35f7-4475-b870-469d659b739d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.392554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad779-5695-417a-b801-3786286eebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.392964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad779-5695-417a-b801-3786286eebda admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.393243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad779-5695-417a-b801-3786286eebda admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.393511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad779-5695-417a-b801-3786286eebda admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.394162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ad779-5695-417a-b801-3786286eebda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.396427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946ad779-5695-417a-b801-3786286eebda admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.405352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f20e17-c8cf-4521-8662-dd23922f6cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.406826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f20e17-c8cf-4521-8662-dd23922f6cad admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.407539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f20e17-c8cf-4521-8662-dd23922f6cad admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.409289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f20e17-c8cf-4521-8662-dd23922f6cad admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.410258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f20e17-c8cf-4521-8662-dd23922f6cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.412555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f20e17-c8cf-4521-8662-dd23922f6cad admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.534066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 913/1804] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:46 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:46.535040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad779-5695-417a-b801-3786286eebda admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.536816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-946ad779-5695-417a-b801-3786286eebda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.556497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad779-5695-417a-b801-3786286eebda admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.584762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:46.585046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.585715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.586370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.586884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.587983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.589406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 892/1805] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:46 2022] POST /identity/v3/users => generated 296 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:46.590796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.591252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.591552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.591902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.605072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 914/1806] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:46.607031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.611999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.625623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:46.627099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.639543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 893/1807] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:46.642303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.653531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f148597-879e-40d4-96f6-c8b9e0ad4a4f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.680712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.686082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.689455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.690064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.700978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.700978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.714326 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 894/1808] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:46.715743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.737159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721c937-b793-4f15-a00d-815cc05af177 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.767398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.768371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.768848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.769309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.770239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.772084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.793284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 895/1809] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:46.794182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.816770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83cbf6-e354-4acb-9493-ea299038ea1a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.823539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.824313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.824603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.824885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] PATH_INFO: `/v3/projects/f6e6edeaee524a8182d02298133c40ff/users/50a96f3604074ba7a00b47e760e9c8e8/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.840094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165cc60-f336-4bd1-9ffd-e5302c19557d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:46.847277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:46.848228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:46.849329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:46.849951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:46.850662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e6edeaee524a8182d02298133c40ff, user_id=50a96f3604074ba7a00b47e760e9c8e8, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.853215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.854224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.943964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:46.945520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:46.955744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 915/1810] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:32:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:32:46.956672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:46.958395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:46.974276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 896/1811] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:46 2022] PUT /identity/v3/projects/f6e6edeaee524a8182d02298133c40ff/users/50a96f3604074ba7a00b47e760e9c8e8/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:46.975402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.086134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:47.102229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd13df-f485-4727-b2a6-c0c731c662c3 service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:47.166253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979b760-8b46-4ea2-8d79-92f4f7eddee5 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:47.187647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:47.187647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.187647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.187647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.188042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:47.191298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:47.227659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 897/1812] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:47.228659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610c447-7047-45f2-a634-bbe48c1d98a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:47.228782 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.229723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610c447-7047-45f2-a634-bbe48c1d98a9 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.231387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610c447-7047-45f2-a634-bbe48c1d98a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.232015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610c447-7047-45f2-a634-bbe48c1d98a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.253268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610c447-7047-45f2-a634-bbe48c1d98a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:47.269228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610c447-7047-45f2-a634-bbe48c1d98a9 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:47.270932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547aa41-8b2c-4238-9723-034450dc761b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:47.285745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:47.286519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.286807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.295604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] PATH_INFO: `/v3/projects/03ee8b035f9e42af825dcf9a69a0f9ee/users/cf469e1808194240950dc11dab138fe7/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.316271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 916/1813] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:47.322728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.323126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.323494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.338144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ee8b035f9e42af825dcf9a69a0f9ee, user_id=cf469e1808194240950dc11dab138fe7, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:47.344560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:47.408498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:47.409762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:47.438625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 917/1814] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Aug 8 00:32:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:47.439611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.467121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91913d7-ebae-48a1-8fbb-f842b3a42b71 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:47.477599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 898/1815] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:47 2022] PUT /identity/v3/projects/03ee8b035f9e42af825dcf9a69a0f9ee/users/cf469e1808194240950dc11dab138fe7/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:47.479168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.579550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:47.580481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.580839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.581344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.582058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:47.584010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:47.586077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6048db-0463-4c2a-98cf-5c3672fc6e9f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:47.592485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:47.593572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.594230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.595435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] PATH_INFO: `/v3/projects/9f8391e35bf14063a7117ee41e770132/users/f86100687383464cac90757ac0e72129/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.614844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 918/1816] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:47.615924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.652926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba066f-80a7-469d-9d7b-4f6e606484ba admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:47.668533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8391e35bf14063a7117ee41e770132, user_id=f86100687383464cac90757ac0e72129, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:47.669059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:47.669694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:47.669976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:47.670254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:47.670949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:47.675514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:47.680124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:47.711295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 919/1817] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:47.711295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.719484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:47.867875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 899/1818] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:47 2022] PUT /identity/v3/projects/9f8391e35bf14063a7117ee41e770132/users/f86100687383464cac90757ac0e72129/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:47.867875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:47.941001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca80831-3993-43f9-a497-726132bfdb61 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:48.047074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724eb08d-1e27-43ff-9b50-3fac1e851c11 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:48.073730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:48.073730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.073730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.073730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.074285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871afa-297b-486b-b623-849493746831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:48.074285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871afa-297b-486b-b623-849493746831 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.074285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871afa-297b-486b-b623-849493746831 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.074285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871afa-297b-486b-b623-849493746831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.082392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db871afa-297b-486b-b623-849493746831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:48.084866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db871afa-297b-486b-b623-849493746831 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:48.087372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:48.088789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:48.106455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 920/1819] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:48.107699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 900/1820] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:48.108556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:48.108961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871afa-297b-486b-b623-849493746831 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:48.303746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b6699-9728-4a95-8c47-5d402c742a2d service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:48.317759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:48.319494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.320315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.322209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] PATH_INFO: `/v3/projects/03ee8b035f9e42af825dcf9a69a0f9ee/users/cf469e1808194240950dc11dab138fe7/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.326506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ee8b035f9e42af825dcf9a69a0f9ee, user_id=cf469e1808194240950dc11dab138fe7, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:48.334809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:48.351199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871afa-297b-486b-b623-849493746831 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:48.373603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 901/1821] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:48 2022] PUT /identity/v3/projects/03ee8b035f9e42af825dcf9a69a0f9ee/users/cf469e1808194240950dc11dab138fe7/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:48.373603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:48.501533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1037-453f-4c25-96a0-d24d67cd9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:48.502565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1037-453f-4c25-96a0-d24d67cd9583 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.502565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1037-453f-4c25-96a0-d24d67cd9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.503173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1037-453f-4c25-96a0-d24d67cd9583 admin admin] PATH_INFO: `/v3/projects/f6e6edeaee524a8182d02298133c40ff/users/50a96f3604074ba7a00b47e760e9c8e8/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.508044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e1037-453f-4c25-96a0-d24d67cd9583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6e6edeaee524a8182d02298133c40ff, user_id=50a96f3604074ba7a00b47e760e9c8e8, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:48.515751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e1037-453f-4c25-96a0-d24d67cd9583 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:48.538105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 921/1822] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:48 2022] PUT /identity/v3/projects/f6e6edeaee524a8182d02298133c40ff/users/50a96f3604074ba7a00b47e760e9c8e8/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:48.539255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6575c-172d-4737-b1d8-bd156a43fb76 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:48.540155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0170-8a9d-4bde-b260-5d998d5cc1f2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.540761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0170-8a9d-4bde-b260-5d998d5cc1f2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.541236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f0170-8a9d-4bde-b260-5d998d5cc1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.589605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6f3f0170-8a9d-4bde-b260-5d998d5cc1f2 None None] MFA Rules not processed for user `f86100687383464cac90757ac0e72129`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:48.593559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6f3f0170-8a9d-4bde-b260-5d998d5cc1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:48.661515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476257b6-2e5d-42ae-ac76-31c8e0164760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:48.661515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476257b6-2e5d-42ae-ac76-31c8e0164760 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.661515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476257b6-2e5d-42ae-ac76-31c8e0164760 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.661515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476257b6-2e5d-42ae-ac76-31c8e0164760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.661940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476257b6-2e5d-42ae-ac76-31c8e0164760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:48.661940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476257b6-2e5d-42ae-ac76-31c8e0164760 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:48.696855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 902/1823] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:48.698352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.698626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.698897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.719292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 922/1824] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:48.721043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:48.721516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:48.722980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:48.764041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:48.789446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] MFA Rules not processed for user `cf469e1808194240950dc11dab138fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:48.794341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:48.799474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] MFA Rules not processed for user `50a96f3604074ba7a00b47e760e9c8e8`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:48.799675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:48.948185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 923/1825] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:48.949215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:48.982022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 903/1826] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:48.989624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:48.997011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:49.064753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:49.085683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041fba5-f26b-4f56-93ad-6fda055b1154 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:49.106663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9bbda-4334-428e-9edc-b0769eac6646 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:49.111689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:49.111689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:49.114634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:49.114634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:49.114634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:49.116482 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:49.129285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:49.131008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:49.131625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:49.137497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] PATH_INFO: `/v3/projects/65fc9fdf93124d50b1c783a9271f1faf` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:49.137497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fc9fdf93124d50b1c783a9271f1faf)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:49.137497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:49.277018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 924/1827] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:48 2022] POST /identity/v3/projects => generated 379 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:49.277018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:49.279856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:49.401390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818097c-cbff-40cc-8938-51c346871b98 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:49.402792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fc9fdf93124d50b1c783a9271f1faf', 'request_id': 'req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:49.424589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fc9fdf93124d50b1c783a9271f1faf', 'request_id': 'req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:49.443133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fc9fdf93124d50b1c783a9271f1faf', 'request_id': 'req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:49.513575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3da9ff7-9832-44f1-ac7f-e16534219f43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:49.513575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da9ff7-9832-44f1-ac7f-e16534219f43 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:49.513575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da9ff7-9832-44f1-ac7f-e16534219f43 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:49.513575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da9ff7-9832-44f1-ac7f-e16534219f43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:49.655206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da9ff7-9832-44f1-ac7f-e16534219f43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:49.673906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3da9ff7-9832-44f1-ac7f-e16534219f43 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:49.719586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] The token cache is being invalidate because project 65fc9fdf93124d50b1c783a9271f1faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:49.723290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:49.723290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:49.723290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 904/1828] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:48 2022] DELETE /identity/v3/projects/65fc9fdf93124d50b1c783a9271f1faf => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:49.725940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:49.731443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:49.736124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 925/1829] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:49.739052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:49.739371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:49.739645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:49.797712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:49.799871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:49.876863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b8db7-37f2-403f-a73a-e8f48b0f8e0b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:49.891587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:49.891587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:49.891587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:49.891587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:49.892474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:49.892474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:49.923856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 926/1830] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:49.923856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:49.926156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:49.954263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62028d04-0caf-4c1a-bdca-421fdaa7fcc0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:49.963911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:49.964775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:49.965214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:49.965668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:49.974561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:50.073154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:50.074632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a991b70-e9e7-4150-9041-2fa35c76f30a service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:50.086714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 927/1831] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:50.088343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:50.088549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:50.088822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:50.112690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 905/1832] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:49 2022] POST /identity/v3/users => generated 296 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:50.114637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:50.119062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:50.120601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:50.133775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 928/1833] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:50.137195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:50.146861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca720f8f-4691-4ae6-8e39-8eb4b8bb1892 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:50.148948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfea124-8fe1-4d21-b06b-64a5db6b8878 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:50.155065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:50.155861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:50.156662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:50.157484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:50.157790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:50.158227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:50.160027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:50.180924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:50.181634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:50.181926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:50.182243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:50.214046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:50.214046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:50.215131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 929/1834] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:50.226446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:50.238055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 906/1835] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:50.257756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41814958-3d3e-479a-856a-31eabe329e42 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:50.410057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:50.410057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:50.410057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:50.410057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:50.412112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:50.412112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:50.412359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:50.459330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7bf23-4a32-49a6-80ce-4cec19cf198b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:50.493017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df24dfe-489c-4370-a5c0-457fd7e4e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:50.493705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df24dfe-489c-4370-a5c0-457fd7e4e868 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:50.493988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df24dfe-489c-4370-a5c0-457fd7e4e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:50.494262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df24dfe-489c-4370-a5c0-457fd7e4e868 admin admin] PATH_INFO: `/v3/projects/564d9d4dc197455b93499c8feaae96d0/users/97704de6b34a4264b7df236727ad606f/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:50.561094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df24dfe-489c-4370-a5c0-457fd7e4e868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564d9d4dc197455b93499c8feaae96d0, user_id=97704de6b34a4264b7df236727ad606f, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:50.564528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df24dfe-489c-4370-a5c0-457fd7e4e868 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:50.584138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 930/1836] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:50 2022] POST /identity/v3/projects => generated 357 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:50.599987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:50.758497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 907/1837] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:50 2022] PUT /identity/v3/projects/564d9d4dc197455b93499c8feaae96d0/users/97704de6b34a4264b7df236727ad606f/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:50.760298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e671b2-5bd0-45c3-a32d-af01a5e7adbe None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:50.760744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e671b2-5bd0-45c3-a32d-af01a5e7adbe None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:50.761400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e671b2-5bd0-45c3-a32d-af01a5e7adbe None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:50.767771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 908/1838] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:32:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:50.773992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e671b2-5bd0-45c3-a32d-af01a5e7adbe None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:50.856779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:50.893582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce10074-ae07-4166-96d7-dde96dd388c3 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.046644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e671b2-5bd0-45c3-a32d-af01a5e7adbe None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.053902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfdfcb-8bef-42ae-b000-4de403d359ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.054208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.054208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.054508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfdfcb-8bef-42ae-b000-4de403d359ff admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.054616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.054916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.055026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfdfcb-8bef-42ae-b000-4de403d359ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.055628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfdfcb-8bef-42ae-b000-4de403d359ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.056027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.056478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfdfcb-8bef-42ae-b000-4de403d359ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.058468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfdfcb-8bef-42ae-b000-4de403d359ff admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.059796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.073536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 909/1839] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:51.100312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27a2e-0d9c-44b6-a0ac-aaa9fe2ee227 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.101356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27a2e-0d9c-44b6-a0ac-aaa9fe2ee227 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.107366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27a2e-0d9c-44b6-a0ac-aaa9fe2ee227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.151159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-60e27a2e-0d9c-44b6-a0ac-aaa9fe2ee227 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:51.154730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-60e27a2e-0d9c-44b6-a0ac-aaa9fe2ee227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:51.238730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 931/1840] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:50 2022] POST /identity/v3/users => generated 279 bytes in 636 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:51.238730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.261511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa257577-4917-4a11-a092-f0821b590ffb admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.291530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e62b1-c56a-47c0-9012-f3709598f548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.291530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e62b1-c56a-47c0-9012-f3709598f548 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.291530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e62b1-c56a-47c0-9012-f3709598f548 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.291530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e62b1-c56a-47c0-9012-f3709598f548 admin admin] PATH_INFO: `/v3/projects/564d9d4dc197455b93499c8feaae96d0/users/97704de6b34a4264b7df236727ad606f/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.295392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e62b1-c56a-47c0-9012-f3709598f548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=564d9d4dc197455b93499c8feaae96d0, user_id=97704de6b34a4264b7df236727ad606f, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.301222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 910/1841] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:32:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:51.302993 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f82a2-c6bd-49a4-8dcc-7824468cac57 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.303637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f82a2-c6bd-49a4-8dcc-7824468cac57 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.304093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f82a2-c6bd-49a4-8dcc-7824468cac57 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.305583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e62b1-c56a-47c0-9012-f3709598f548 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.305789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 911/1842] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:32:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:32:51.306549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f82a2-c6bd-49a4-8dcc-7824468cac57 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.328508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f82a2-c6bd-49a4-8dcc-7824468cac57 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.330094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 932/1843] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:51 2022] PUT /identity/v3/projects/564d9d4dc197455b93499c8feaae96d0/users/97704de6b34a4264b7df236727ad606f/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:51.337504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.337887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.338259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.385324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:51.386935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:51.420275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131ead4-195c-4a21-af7a-f090008ca81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.420881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131ead4-195c-4a21-af7a-f090008ca81e admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.421697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131ead4-195c-4a21-af7a-f090008ca81e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.421697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131ead4-195c-4a21-af7a-f090008ca81e admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.422096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131ead4-195c-4a21-af7a-f090008ca81e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.424364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131ead4-195c-4a21-af7a-f090008ca81e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.436029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 912/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:51.437688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.437688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.437976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.467530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] MFA Rules not processed for user `97704de6b34a4264b7df236727ad606f`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:51.470428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 933/1845] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:32:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:51.471300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.480185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:51.483131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:51.503572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358009d2-8b29-43cd-b6a4-64486d3f7c95 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.509605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.510233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.510515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.510784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.521470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.523434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.537989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 934/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:51.538461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.559912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3831f-d019-4d49-8e1c-92ee1536ebac service nova] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.566060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.566235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.566439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.566715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.567395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.569028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.584519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 935/1847] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:51.584519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.587469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:51.591341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 913/1848] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:51.592254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.603434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aa0fe-b35b-4277-8919-74e4f4149ddb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.607634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a1378-6c70-4fbc-8cee-a9591e6630c7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.608792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea95fe-4f7e-44d9-9bc2-8d29946b8562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.609465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea95fe-4f7e-44d9-9bc2-8d29946b8562 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.609719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea95fe-4f7e-44d9-9bc2-8d29946b8562 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.609998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea95fe-4f7e-44d9-9bc2-8d29946b8562 admin admin] PATH_INFO: `/v3/projects/f3b7101185d64cdd9028082404030a03/users/8e7a76bc1ec0409399318f8b609db224/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.618310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.619021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.619593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.619873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.631004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.632667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea95fe-4f7e-44d9-9bc2-8d29946b8562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b7101185d64cdd9028082404030a03, user_id=8e7a76bc1ec0409399318f8b609db224, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.632790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.635541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea95fe-4f7e-44d9-9bc2-8d29946b8562 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.646647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 936/1849] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:51.647971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:51.673659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.731564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 914/1850] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:51 2022] PUT /identity/v3/projects/f3b7101185d64cdd9028082404030a03/users/8e7a76bc1ec0409399318f8b609db224/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:51.742561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.742970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.743354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.748960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca237c7-efc0-4529-87d4-4c5d5bbd07b3 service nova] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.775584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] MFA Rules not processed for user `8e7a76bc1ec0409399318f8b609db224`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:51.777794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:51.808067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.809024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.809296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.809568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.869437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 915/1851] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:51.878931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.887340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.888781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:51.907123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 937/1852] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:51.941072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:51.949418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3c0f-2867-4a72-9560-3bf442d21c70 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:51.954907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:51.957119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:51.957119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:51.957119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:51.957119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:51.959144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.005861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee522d7-033f-4768-87cd-95c8563d5c54 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.010754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf469e1808194240950dc11dab138fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03ee8b035f9e42af825dcf9a69a0f9ee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.015341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.015341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.015341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.015341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:52.015341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.015780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.026144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 938/1853] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:32:52.030233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.044169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39af06-e00e-464a-a164-441d5f8cc579 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.044406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf469e1808194240950dc11dab138fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03ee8b035f9e42af825dcf9a69a0f9ee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.044997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.045278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.045552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.046650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.048686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.103269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 916/1854] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:51 2022] POST /identity/v3/projects => generated 357 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:52.106984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.165029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 939/1855] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:52 2022] POST /identity/v3/projects => generated 354 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:52.177829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.223874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4deffe-2137-4825-86a6-535f65ef81b4 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.304519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819185aa-e011-4fc4-aa94-ca3e0e82f8c9 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.318354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf469e1808194240950dc11dab138fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03ee8b035f9e42af825dcf9a69a0f9ee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.319066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.319394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.319754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.320473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:52.320923 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.323518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.338574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 940/1856] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:52.350830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.351741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.353320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.355257 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.355996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.358053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.417572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.430239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21c869-11dd-49d7-a21c-2d788a9dea34 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.435938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf469e1808194240950dc11dab138fe7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '03ee8b035f9e42af825dcf9a69a0f9ee', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.436579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.437849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.438135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] PATH_INFO: `/v3/projects/f935b31b09194aa99bb46c0d9d2f305e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.439297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f935b31b09194aa99bb46c0d9d2f305e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.441281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.513985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 917/1857] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:52 2022] POST /identity/v3/users => generated 285 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:52.518259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.528389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e526e2e-7915-4e4e-9402-66295d795d79 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.534009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.534629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.534916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.535181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.540088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.540088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.551253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 918/1858] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:52.554423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.561373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f935b31b09194aa99bb46c0d9d2f305e', 'request_id': 'req-9917bb17-292a-489c-b253-41930e0ccb98'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:52.565527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedfe58-f4ab-498f-912c-1faf902aec50 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.567685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f935b31b09194aa99bb46c0d9d2f305e', 'request_id': 'req-9917bb17-292a-489c-b253-41930e0ccb98'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:52.575175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f935b31b09194aa99bb46c0d9d2f305e', 'request_id': 'req-9917bb17-292a-489c-b253-41930e0ccb98'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:52.621071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.621904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.622144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.622468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] PATH_INFO: `/v3/projects/a1d1b2aedf174fd7b0ddd2f974d1f007/users/e98db5c482ea49d6b2369b14a581088e/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.640470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d1b2aedf174fd7b0ddd2f974d1f007, user_id=e98db5c482ea49d6b2369b14a581088e, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.644098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.681296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] The token cache is being invalidate because project f935b31b09194aa99bb46c0d9d2f305e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:52.681607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:52.682396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:52.688331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 941/1859] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:52 2022] DELETE /identity/v3/projects/f935b31b09194aa99bb46c0d9d2f305e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:52.688331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.688646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:52.710322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917bb17-292a-489c-b253-41930e0ccb98 tempest-VolumeQuotaClassesTest-216478907 tempest-VolumeQuotaClassesTest-216478907-project-admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.719493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.726647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.726647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.726647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] PATH_INFO: `/v3/users/f86100687383464cac90757ac0e72129` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.726647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86100687383464cac90757ac0e72129)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.726647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.778624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 919/1860] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:52 2022] PUT /identity/v3/projects/a1d1b2aedf174fd7b0ddd2f974d1f007/users/e98db5c482ea49d6b2369b14a581088e/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:52.784411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.786189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:52.858377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa5577-ffa2-4da3-936d-6582714fd925 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.863133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.863767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.864052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.864322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.864962 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.866623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.887632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 920/1861] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:52.895211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:52.925216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8902935-484d-4f9e-a659-37c95a8cfb3d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:52.934977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1870622-bf3b-4e38-9f40-18ec6fc05f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:52.936888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1870622-bf3b-4e38-9f40-18ec6fc05f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.937312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1870622-bf3b-4e38-9f40-18ec6fc05f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.937821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1870622-bf3b-4e38-9f40-18ec6fc05f8c admin admin] PATH_INFO: `/v3/projects/a1d1b2aedf174fd7b0ddd2f974d1f007/users/e98db5c482ea49d6b2369b14a581088e/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:52.942786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1870622-bf3b-4e38-9f40-18ec6fc05f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d1b2aedf174fd7b0ddd2f974d1f007, user_id=e98db5c482ea49d6b2369b14a581088e, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:52.945591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1870622-bf3b-4e38-9f40-18ec6fc05f8c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:52.964248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 921/1862] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:52 2022] PUT /identity/v3/projects/a1d1b2aedf174fd7b0ddd2f974d1f007/users/e98db5c482ea49d6b2369b14a581088e/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:52.980526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082858-0f85-463b-af7e-2c35a275e555 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:52.981280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082858-0f85-463b-af7e-2c35a275e555 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:52.981280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082858-0f85-463b-af7e-2c35a275e555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:53.021116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ca082858-0f85-463b-af7e-2c35a275e555 None None] MFA Rules not processed for user `e98db5c482ea49d6b2369b14a581088e`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:53.021116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ca082858-0f85-463b-af7e-2c35a275e555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:53.064068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86100687383464cac90757ac0e72129'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:53.100819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86100687383464cac90757ac0e72129'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:53.102553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86100687383464cac90757ac0e72129'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:53.138446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86100687383464cac90757ac0e72129'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:53.168070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 922/1863] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:32:53.444572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86100687383464cac90757ac0e72129'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:53.450978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 942/1864] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:52 2022] DELETE /identity/v3/users/f86100687383464cac90757ac0e72129 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:53.456414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:53.456709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:53.456987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:53.483210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:53.484743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:53.485114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:53.486436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:53.587600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 923/1865] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:53.604526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:53.615292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:53.689586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7521c3d-8683-4258-8178-d59bce443a09 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:53.689586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:53.689586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:53.691539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:53.691941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:53.693990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:53.740832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361595a-5e8a-4d7f-a6d3-09a770981c6e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:53.764221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:53.765445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:53.765774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:53.766029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:53.779010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:53.781159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:53.807987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 924/1866] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:53.813636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:53.815206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:53.827011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 943/1867] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:53.869286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:53.909900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:53.911452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:53.941071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7deeaa-fbe5-4b6f-97f3-f3177c81bf93 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:53.959030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:53.959030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:53.959030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:53.959030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] PATH_INFO: `/v3/users/60136c1889d0478b86ab68b4131b0b97` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:53.959662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60136c1889d0478b86ab68b4131b0b97)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:53.961659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:54.365218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60136c1889d0478b86ab68b4131b0b97'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:54.391095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60136c1889d0478b86ab68b4131b0b97'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:54.391095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60136c1889d0478b86ab68b4131b0b97'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:54.432577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60136c1889d0478b86ab68b4131b0b97'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:54.433427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:54.442065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0935ea-e190-4823-8102-30a64670caa8 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:54.445849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:54.446479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:54.446760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:54.447031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:54.448813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:54.520731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:54.522564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538be079-e016-4f83-98c4-df4c9eaa9005 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:54.538022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 944/1868] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:54 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:32:54.661541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60136c1889d0478b86ab68b4131b0b97'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:54.678830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 925/1869] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:53 2022] DELETE /identity/v3/users/60136c1889d0478b86ab68b4131b0b97 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:54.680884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:54.681172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:54.681444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:54.706094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:54.709187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:54.757980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:54.762561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:54.845277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040245d1-4316-4ac4-a12c-5f2510390a68 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:54.851158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:54.851827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:54.852091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:54.852367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] PATH_INFO: `/v3/projects/9f8391e35bf14063a7117ee41e770132` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:54.853751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8391e35bf14063a7117ee41e770132)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:54.856344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:54.866264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 945/1870] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:54.878145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:54.879936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:54.951289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4716c-15b9-4cd0-8a36-0fee207e7a44 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:54.956621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:54.957959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:54.958733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:54.959187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:54.970655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:54.973855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:54.990392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 946/1871] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:55.023682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8391e35bf14063a7117ee41e770132', 'request_id': 'req-1727cd05-fd76-48d4-9699-a8e01f0edd60'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:55.035296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8391e35bf14063a7117ee41e770132', 'request_id': 'req-1727cd05-fd76-48d4-9699-a8e01f0edd60'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:55.044546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8391e35bf14063a7117ee41e770132', 'request_id': 'req-1727cd05-fd76-48d4-9699-a8e01f0edd60'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:55.292836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] The token cache is being invalidate because project 9f8391e35bf14063a7117ee41e770132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:55.293133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:55.293873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:55.295311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 926/1872] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:54 2022] DELETE /identity/v3/projects/9f8391e35bf14063a7117ee41e770132 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:55.300028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:55.314490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e9d15-5480-4248-bfc4-a78688e5a39e service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:55.387108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:55.394391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:55.394391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:55.394391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] PATH_INFO: `/v3/users/97704de6b34a4264b7df236727ad606f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:55.395060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97704de6b34a4264b7df236727ad606f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:55.397402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:55.976347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97704de6b34a4264b7df236727ad606f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.032074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97704de6b34a4264b7df236727ad606f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.036121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97704de6b34a4264b7df236727ad606f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.045628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:56.047774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:56.072450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97704de6b34a4264b7df236727ad606f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.087029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727cd05-fd76-48d4-9699-a8e01f0edd60 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:56.101865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:56.102834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:56.103867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:56.104258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] PATH_INFO: `/v3/projects/8191bccb93da4bf5ad496f82641cfeae` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:56.105828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8191bccb93da4bf5ad496f82641cfeae)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:56.121110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:56.151848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97704de6b34a4264b7df236727ad606f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.164445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 947/1873] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:55 2022] DELETE /identity/v3/users/97704de6b34a4264b7df236727ad606f => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:32:56.207847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:32:56.582498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8191bccb93da4bf5ad496f82641cfeae', 'request_id': 'req-591edae8-4dfe-44be-b2e7-991fff2ab6c7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.588142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8191bccb93da4bf5ad496f82641cfeae', 'request_id': 'req-591edae8-4dfe-44be-b2e7-991fff2ab6c7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.601378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8191bccb93da4bf5ad496f82641cfeae', 'request_id': 'req-591edae8-4dfe-44be-b2e7-991fff2ab6c7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.710543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] The token cache is being invalidate because project 8191bccb93da4bf5ad496f82641cfeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:56.710943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.712276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:56.713901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 927/1874] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:56 2022] DELETE /identity/v3/projects/8191bccb93da4bf5ad496f82641cfeae => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:56.719337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:56.721321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:56.820214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d08490-1478-436e-8a99-7262dbd857c7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:56.898559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:56.899212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:56.899527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:56.904129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] PATH_INFO: `/v3/users/cf469e1808194240950dc11dab138fe7` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:56.907744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf469e1808194240950dc11dab138fe7)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:56.910057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:56.930579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:56.932495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:56.977947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591edae8-4dfe-44be-b2e7-991fff2ab6c7 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:56.990123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:56.991008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:56.991540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:56.992017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] PATH_INFO: `/v3/projects/564d9d4dc197455b93499c8feaae96d0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:56.993466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=564d9d4dc197455b93499c8feaae96d0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:56.995724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:57.480809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf469e1808194240950dc11dab138fe7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.492124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564d9d4dc197455b93499c8feaae96d0', 'request_id': 'req-880d43ef-2cc9-44e5-96fe-9b7d06897f09'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.503308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf469e1808194240950dc11dab138fe7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.504324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf469e1808194240950dc11dab138fe7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.516990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf469e1808194240950dc11dab138fe7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.524135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564d9d4dc197455b93499c8feaae96d0', 'request_id': 'req-880d43ef-2cc9-44e5-96fe-9b7d06897f09'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.550986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564d9d4dc197455b93499c8feaae96d0', 'request_id': 'req-880d43ef-2cc9-44e5-96fe-9b7d06897f09'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.647544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf469e1808194240950dc11dab138fe7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.660059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 948/1875] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:56 2022] DELETE /identity/v3/users/cf469e1808194240950dc11dab138fe7 => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:57.661980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:57.665332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:57.808481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdfbe1f-5f1d-46d0-b737-51c407b9dea7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:57.815516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:57.816401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:57.816831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:57.817251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] PATH_INFO: `/v3/users/909e20a8b2a24f759dfe9fe33dcbf56f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:57.819193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=909e20a8b2a24f759dfe9fe33dcbf56f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:57.821753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:57.851575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] The token cache is being invalidate because project 564d9d4dc197455b93499c8feaae96d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:32:57.852088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.853333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-880d43ef-2cc9-44e5-96fe-9b7d06897f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:57.855741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 928/1876] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:56 2022] DELETE /identity/v3/projects/564d9d4dc197455b93499c8feaae96d0 => generated 0 bytes in 926 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:57.861683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f46ba-d905-47aa-a68d-a0beb98881e1 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:57.862261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f46ba-d905-47aa-a68d-a0beb98881e1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:57.862804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f46ba-d905-47aa-a68d-a0beb98881e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:57.893629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cf8f46ba-d905-47aa-a68d-a0beb98881e1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:57.895612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cf8f46ba-d905-47aa-a68d-a0beb98881e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:57.909437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 929/1877] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:57.927327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:57.928996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:57.931608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:57.980980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:57.982450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:57.999795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 930/1878] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:58.004879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.024898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b59e32-4b9b-4345-ab18-7e26b9780494 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.033763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.034446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.034804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.034990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.035729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:32:58.038989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.038989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.048673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '909e20a8b2a24f759dfe9fe33dcbf56f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:58.073921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 931/1879] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:32:58.078372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.095332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d37b1a-66a7-4448-bb31-c7e683361f2f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.100751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '909e20a8b2a24f759dfe9fe33dcbf56f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:58.102173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '909e20a8b2a24f759dfe9fe33dcbf56f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:58.105620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.106561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.107058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.107603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.110016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.112284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.122972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '909e20a8b2a24f759dfe9fe33dcbf56f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:58.200090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-226f79df-9309-4187-a6ef-9a914e8ce34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '909e20a8b2a24f759dfe9fe33dcbf56f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:58.210688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 949/1880] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:32:57 2022] DELETE /identity/v3/users/909e20a8b2a24f759dfe9fe33dcbf56f => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:58.227136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512712b7-6153-4d9b-a125-d3164196065e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.227136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512712b7-6153-4d9b-a125-d3164196065e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.227136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512712b7-6153-4d9b-a125-d3164196065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.257217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 932/1881] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:58 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:58.262928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.272238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:58.293012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-512712b7-6153-4d9b-a125-d3164196065e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:58.294373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-512712b7-6153-4d9b-a125-d3164196065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:58.427588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34de3b-8a56-4d0a-90ac-b1fdff76f989 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.445446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 950/1882] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:58.450812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.451647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.451924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.453351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.455393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.458803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.547149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 933/1883] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:58 2022] POST /identity/v3/users => generated 292 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:58.547844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512712b7-6153-4d9b-a125-d3164196065e None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.565032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512712b7-6153-4d9b-a125-d3164196065e None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.574302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.575540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.575937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.576294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.577346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.579335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.597676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.609440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 951/1884] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:58.610315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b74ee8-8bcc-4db8-8c44-5bf298cae3fe admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.615190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.615838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.616114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.616456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.617812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.619979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.639352 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.645756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 934/1885] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:32:58.653901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3791b-e551-4ea9-b767-2765c7a159e8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.659925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:58.662291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:58.662413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.662995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.663442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.663723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] PATH_INFO: `/v3/projects/2aab47ede22a437fb8b8aa4333037bb4/users/860de489d3924a32853a3810c3bf1541/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.686551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aab47ede22a437fb8b8aa4333037bb4, user_id=860de489d3924a32853a3810c3bf1541, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.691491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.776498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b21601-b317-4289-8236-7dce3c3457e9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:58.787901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25c991-5e41-40f9-8e0f-c53ed0d25343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:58.787901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25c991-5e41-40f9-8e0f-c53ed0d25343 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.787901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25c991-5e41-40f9-8e0f-c53ed0d25343 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.787901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25c991-5e41-40f9-8e0f-c53ed0d25343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.803722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25c991-5e41-40f9-8e0f-c53ed0d25343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:58.805422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25c991-5e41-40f9-8e0f-c53ed0d25343 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:58.831928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 935/1886] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:32:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:32:58.852157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 952/1887] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:58 2022] PUT /identity/v3/projects/2aab47ede22a437fb8b8aa4333037bb4/users/860de489d3924a32853a3810c3bf1541/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:58.858270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:58.859078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:58.859408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:58.894772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] MFA Rules not processed for user `860de489d3924a32853a3810c3bf1541`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:32:58.896079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:59.131167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 936/1888] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:32:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:32:59.131683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:59.302313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491688-03e7-4797-8f2f-7e55720e2fbf admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:59.316373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:59.317008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:59.317283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:59.317554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:59.318189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:59.320255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:59.487016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 953/1889] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:32:59 2022] POST /identity/v3/projects => generated 381 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:32:59.491196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:59.541790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f72b99-ce91-46e7-983b-a51db1874eab None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:59.648824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:59.648824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:59.648824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:59.648824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:59.649985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:59.651911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:59.720213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:59.721992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:32:59.742791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 937/1890] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:32:59 2022] POST /identity/v3/users => generated 297 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:32:59.747186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:59.752203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654af7e-8ff0-47eb-9305-a36b4262a75f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:59.764643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e512f1-dbdf-4659-a7a3-eefcb270920b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:59.776642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:59.777820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:59.778053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:59.778330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] PATH_INFO: `/v3/projects/03ee8b035f9e42af825dcf9a69a0f9ee` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:59.780213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ee8b035f9e42af825dcf9a69a0f9ee)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:59.783992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:59.784618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:59.785349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:59.785349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:59.785989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:59.788959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:59.789822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:59.809454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 938/1891] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:32:59.816701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:59.840760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936ff22-6ca5-48f4-bc40-c03a3aa209ba admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:32:59.840760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:32:59.840760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:32:59.840760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:32:59.841263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] PATH_INFO: `/v3/projects/5360336ba40f4d77a585b5c79f1bddc8/users/b37556f1507a4b37bf5ce128a38090dc/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:32:59.851856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5360336ba40f4d77a585b5c79f1bddc8, user_id=b37556f1507a4b37bf5ce128a38090dc, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:32:59.854798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:32:59.972348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 939/1892] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:32:59 2022] PUT /identity/v3/projects/5360336ba40f4d77a585b5c79f1bddc8/users/b37556f1507a4b37bf5ce128a38090dc/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:32:59.973336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ee8b035f9e42af825dcf9a69a0f9ee', 'request_id': 'req-de4c3488-ac62-45db-8fe4-c1dd293ffe75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:59.973809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:32:59.987278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ee8b035f9e42af825dcf9a69a0f9ee', 'request_id': 'req-de4c3488-ac62-45db-8fe4-c1dd293ffe75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:32:59.996356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ee8b035f9e42af825dcf9a69a0f9ee', 'request_id': 'req-de4c3488-ac62-45db-8fe4-c1dd293ffe75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.050094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6e04c-b756-4dce-9b7b-d40fa6dc2e2e admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.123961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] The token cache is being invalidate because project 03ee8b035f9e42af825dcf9a69a0f9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:00.125407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.126632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-de4c3488-ac62-45db-8fe4-c1dd293ffe75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.131305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 954/1893] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:32:59 2022] DELETE /identity/v3/projects/03ee8b035f9e42af825dcf9a69a0f9ee => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:00.131452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.131452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.131452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.131725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.132272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.133932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.136750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.137033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.137310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.147878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 940/1894] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:00.155133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.158097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.160392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:00.162390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.182328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a5953-20b3-45d6-b3fc-2ddebd6e2542 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.188216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01bd72-98ad-40f0-afe1-1639678760d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.189049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01bd72-98ad-40f0-afe1-1639678760d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.189403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01bd72-98ad-40f0-afe1-1639678760d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.189747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01bd72-98ad-40f0-afe1-1639678760d5 admin admin] PATH_INFO: `/v3/projects/5360336ba40f4d77a585b5c79f1bddc8/users/b37556f1507a4b37bf5ce128a38090dc/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.196523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 955/1895] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:00.197713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.214863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01bd72-98ad-40f0-afe1-1639678760d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5360336ba40f4d77a585b5c79f1bddc8, user_id=b37556f1507a4b37bf5ce128a38090dc, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.216265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01bd72-98ad-40f0-afe1-1639678760d5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.218406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.239063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37450c70-6858-4bc8-88d8-c4067a7403c5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.242035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 941/1896] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:00 2022] PUT /identity/v3/projects/5360336ba40f4d77a585b5c79f1bddc8/users/b37556f1507a4b37bf5ce128a38090dc/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:00.243200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ef5b-4a44-4920-886d-218cd863d0e6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.247045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ef5b-4a44-4920-886d-218cd863d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.247045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620ef5b-4a44-4920-886d-218cd863d0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.268343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2620ef5b-4a44-4920-886d-218cd863d0e6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:00.274269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2620ef5b-4a44-4920-886d-218cd863d0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.347584 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.349096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.349380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.349654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] PATH_INFO: `/v3/projects/cc1ca37a67064d1f9a40297f7c5722ff` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.350935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1ca37a67064d1f9a40297f7c5722ff)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.354118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.355280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 942/1897] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:00.360654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.360938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.361633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.395256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] MFA Rules not processed for user `b37556f1507a4b37bf5ce128a38090dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:00.396659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.415089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:00.491251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1ca37a67064d1f9a40297f7c5722ff', 'request_id': 'req-7c2601b6-0fc1-4439-9f65-0443d907c9f6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.499680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 943/1898] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:00.500808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.504489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1ca37a67064d1f9a40297f7c5722ff', 'request_id': 'req-7c2601b6-0fc1-4439-9f65-0443d907c9f6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.517975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1ca37a67064d1f9a40297f7c5722ff', 'request_id': 'req-7c2601b6-0fc1-4439-9f65-0443d907c9f6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.525923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867dd526-2252-4d6e-8bc5-12fff27fe02c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.588863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.589477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.589758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.590054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.590788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:00.591070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.592655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.615311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 944/1899] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:00.615635 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.617447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.641616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] The token cache is being invalidate because project cc1ca37a67064d1f9a40297f7c5722ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:00.641909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.642559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:00.644110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 956/1900] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:00 2022] DELETE /identity/v3/projects/cc1ca37a67064d1f9a40297f7c5722ff => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:00.644854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.646126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.668406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2601b6-0fc1-4439-9f65-0443d907c9f6 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.680140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.680758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.681033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.681302 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.681927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.682801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60597fa1-05a5-443f-ad83-6bc30a45ba6d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.683938 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.686798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.687442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.687724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.688024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.689900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.754309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.755862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.766242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 945/1901] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:00.769781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.773581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:00.799446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01914075-9d0e-4926-8958-443f428a6900 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.805593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.807793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.807901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 957/1902] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:00 2022] POST /identity/v3/projects => generated 359 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:00.808328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.808328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] PATH_INFO: `/v3/users/50a96f3604074ba7a00b47e760e9c8e8` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.808574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a96f3604074ba7a00b47e760e9c8e8)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.810162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:00.811012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:00.893269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891509a-928e-40d3-9bc7-e583b22a9370 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:00.898212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:00.898836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:00.899120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:00.899434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:00.900204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:00.905959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.073625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 958/1903] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:00 2022] POST /identity/v3/users => generated 280 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:01.073625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.073625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:01.134147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d1cfd-71aa-44a7-b695-b26f301f2302 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.137927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.138529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.138806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.139074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.141415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:01.157209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a96f3604074ba7a00b47e760e9c8e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:01.172672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a96f3604074ba7a00b47e760e9c8e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:01.173331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a96f3604074ba7a00b47e760e9c8e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:01.180433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a96f3604074ba7a00b47e760e9c8e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:01.203616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.205113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.214590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 959/1904] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:01.215519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.226447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4cc77-44f0-4b33-9650-7a005000871d service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.236026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.236610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.236945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.237215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.239287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.239850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.250650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 960/1905] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:01.255398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.263170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a96f3604074ba7a00b47e760e9c8e8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:01.267720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e260a-0db9-4ade-84e1-0cdc0f8d8d37 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.273683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a458cf33-0946-49da-b047-431f87a077ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.274469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458cf33-0946-49da-b047-431f87a077ca admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.274751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458cf33-0946-49da-b047-431f87a077ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.275021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a458cf33-0946-49da-b047-431f87a077ca admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.276009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458cf33-0946-49da-b047-431f87a077ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.277430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 946/1906] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:00 2022] DELETE /identity/v3/users/50a96f3604074ba7a00b47e760e9c8e8 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:01.279919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a458cf33-0946-49da-b047-431f87a077ca admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.300688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 961/1907] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:01.304882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.306445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:01.378491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:01.391445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df102d0d-bd90-430b-873f-3cb8212a6dfb admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.391445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.393479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.393479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.393479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] PATH_INFO: `/v3/projects/fff526044867472b8a863606aeabaf21/users/acf0dcae3d5640579a5dc580e108e3be/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.410790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff526044867472b8a863606aeabaf21, user_id=acf0dcae3d5640579a5dc580e108e3be, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.413805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.508078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 947/1908] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:01 2022] PUT /identity/v3/projects/fff526044867472b8a863606aeabaf21/users/acf0dcae3d5640579a5dc580e108e3be/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:01.519117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.519564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.519760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.550227 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] MFA Rules not processed for user `acf0dcae3d5640579a5dc580e108e3be`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:01.551585 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:01.691890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 962/1909] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:01.771566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.773753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:01.797350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.799591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:01.843992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365057c-09f1-4693-84f8-ea6f1441f023 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.852183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.852792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.853068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.853336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.861873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.865371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.872235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 948/1910] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:01.881969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105426e8-7a84-42d1-9cf4-f4329724a31d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.887798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:01.893115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.893115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.893115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.893115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] PATH_INFO: `/v3/users/860de489d3924a32853a3810c3bf1541` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.893922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=860de489d3924a32853a3810c3bf1541)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.895996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:01.899792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c70ea-0bcd-40e4-9bf4-cb092fd11653 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:01.904610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:01.905252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:01.906865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:01.907142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] PATH_INFO: `/v3/users/acf0dcae3d5640579a5dc580e108e3be` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:01.908336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf0dcae3d5640579a5dc580e108e3be)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:01.910249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:02.144925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860de489d3924a32853a3810c3bf1541'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.150238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf0dcae3d5640579a5dc580e108e3be'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.157028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860de489d3924a32853a3810c3bf1541'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.158565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860de489d3924a32853a3810c3bf1541'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.164614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860de489d3924a32853a3810c3bf1541'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.166847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf0dcae3d5640579a5dc580e108e3be'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.167693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf0dcae3d5640579a5dc580e108e3be'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.175568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf0dcae3d5640579a5dc580e108e3be'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.229820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860de489d3924a32853a3810c3bf1541'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.236336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 963/1911] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:01 2022] DELETE /identity/v3/users/860de489d3924a32853a3810c3bf1541 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:02.237129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:02.238917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.302484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea129-92c7-4228-93f6-42b23e67b5f9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:02.307343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:02.307975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.308356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.308639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] PATH_INFO: `/v3/projects/f6e6edeaee524a8182d02298133c40ff` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.309759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6e6edeaee524a8182d02298133c40ff)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:02.312233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:02.318391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-13ff01d8-8651-4477-91a7-a94a5f7fd512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf0dcae3d5640579a5dc580e108e3be'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.324713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 949/1912] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:01 2022] DELETE /identity/v3/users/acf0dcae3d5640579a5dc580e108e3be => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:02.326118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca0a0f-f4e7-487e-9f97-b4d3addfb358 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.326294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca0a0f-f4e7-487e-9f97-b4d3addfb358 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.326533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca0a0f-f4e7-487e-9f97-b4d3addfb358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.345733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-adca0a0f-f4e7-487e-9f97-b4d3addfb358 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:02.346959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-adca0a0f-f4e7-487e-9f97-b4d3addfb358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.408321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 950/1913] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:02.410134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.410606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.410883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.415080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6e6edeaee524a8182d02298133c40ff', 'request_id': 'req-2c91044a-443c-41c0-a290-aa12f1c615ad'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.423983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6e6edeaee524a8182d02298133c40ff', 'request_id': 'req-2c91044a-443c-41c0-a290-aa12f1c615ad'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.430497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6e6edeaee524a8182d02298133c40ff', 'request_id': 'req-2c91044a-443c-41c0-a290-aa12f1c615ad'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.431960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:02.433797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.497125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 951/1914] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:02.497905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:02.502788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.563060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] The token cache is being invalidate because project f6e6edeaee524a8182d02298133c40ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:02.564887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.565991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:02.566602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269d099-3466-4156-9ac9-2f911525c21f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:02.571622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 964/1915] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:02 2022] DELETE /identity/v3/projects/f6e6edeaee524a8182d02298133c40ff => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:02.571622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:02.573955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.577783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:02.578513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.579417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.579417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.582768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.596279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91044a-443c-41c0-a290-aa12f1c615ad admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:02.603489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:02.605062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa62d36-bde1-4860-b1e5-e0399ee26316 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:02.606131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:02.607774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.608276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.609680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.612102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.617955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 952/1916] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:02.618921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.619197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.619551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.641634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:02.643320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6595e1ba-d71d-4bd6-82d2-311180fbf371 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:02.651991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:02.652934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.662265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 965/1917] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:02.668614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 953/1918] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:02.682499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.682826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.683098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.712019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:02.713329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:02.723057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 966/1919] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:02.727210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:02.738948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500d379-ddc3-4d7a-bc34-798d3e718bde None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:02.744042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:02.744823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.745227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.745573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.746404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:02.746870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:02.748559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:02.763063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 954/1920] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:02.769753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:02.786925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623cde64-2def-4946-b68e-7e4f5a80e832 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:02.798566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:02.799648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.799648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.799849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.800417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:02.802328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:02.909850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 967/1921] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:02 2022] POST /identity/v3/projects => generated 391 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:02.915337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:02.925681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da69dd-98e2-4aba-957d-e347a00b9cde admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:02.982982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:02.988399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:02.988399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:02.988399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:02.988399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:02.988399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.102953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 955/1922] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:02 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:03.104287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.124749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b6361-b0c3-4c1a-b5d7-3fc751f9b1f3 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:03.130877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:03.131692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.132142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.133079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.135107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:03.135749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:03.138262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.161448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 968/1923] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:03.167714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.184739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef7b4c1-3e3c-468a-b3b3-58a2e59fbf77 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:03.190189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c3258-3cb9-42f9-9f6d-c47123430737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:03.190971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c3258-3cb9-42f9-9f6d-c47123430737 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.191445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c3258-3cb9-42f9-9f6d-c47123430737 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.191600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c3258-3cb9-42f9-9f6d-c47123430737 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.192534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c3258-3cb9-42f9-9f6d-c47123430737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:03.194983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c3258-3cb9-42f9-9f6d-c47123430737 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.198075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:03.210371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 956/1924] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:03.215339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.227189 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c665-57e7-406e-8fda-159cf4927f28 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:03.232043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:03.232703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.232986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.233265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] PATH_INFO: `/v3/projects/30dca8d1c34242aa8b4c4fe638d85a53/users/4022f60ce6db4f3a9419bc4ed16ba643/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.249770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30dca8d1c34242aa8b4c4fe638d85a53, user_id=4022f60ce6db4f3a9419bc4ed16ba643, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:03.252683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.342579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 969/1925] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:03 2022] PUT /identity/v3/projects/30dca8d1c34242aa8b4c4fe638d85a53/users/4022f60ce6db4f3a9419bc4ed16ba643/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:03.353597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.353877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.354150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.382320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] MFA Rules not processed for user `4022f60ce6db4f3a9419bc4ed16ba643`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:03.383585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:03.471862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 957/1926] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:03.476882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.535219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.537444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:03.565294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17800484-8560-455e-aee4-279a2718acd9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:03.571886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:03.572890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.573183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.573459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.574098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:03.576082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.625293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c64dfa-75e2-4df1-8549-9d3f370b9e6e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:03.630228 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:03.630967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.635584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.635584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] PATH_INFO: `/v3/projects/2aab47ede22a437fb8b8aa4333037bb4` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.635584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aab47ede22a437fb8b8aa4333037bb4)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:03.635584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.695730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 970/1927] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:03 2022] POST /identity/v3/projects => generated 393 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:03.705273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.709915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:03.810547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aab47ede22a437fb8b8aa4333037bb4', 'request_id': 'req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:03.822143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aab47ede22a437fb8b8aa4333037bb4', 'request_id': 'req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:03.837784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aab47ede22a437fb8b8aa4333037bb4', 'request_id': 'req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:03.840039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21209b-dfb5-45c6-8bfb-e7aa240642dd admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:03.957964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] The token cache is being invalidate because project 2aab47ede22a437fb8b8aa4333037bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:03.959041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:03.960198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:03.962706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:03.963398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 958/1928] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:03 2022] DELETE /identity/v3/projects/2aab47ede22a437fb8b8aa4333037bb4 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:03.963981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:03.964595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:03.965206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:03.966064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] PATH_INFO: `/v3/projects/fff526044867472b8a863606aeabaf21` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:03.967437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:03.969136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff526044867472b8a863606aeabaf21)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:03.973334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:03.996717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d5a1d-6d82-4826-bbf3-8ee46688e9d3 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.016510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.017144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.017423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.018171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.018518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.020741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.130911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 959/1929] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:03 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:04.130911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.134806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.150844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff526044867472b8a863606aeabaf21', 'request_id': 'req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.154302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85642937-3215-499b-97e9-52954c0efb2f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.161050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff526044867472b8a863606aeabaf21', 'request_id': 'req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.171190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff526044867472b8a863606aeabaf21', 'request_id': 'req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.275161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] The token cache is being invalidate because project fff526044867472b8a863606aeabaf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:04.275161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.275161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.277837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 971/1930] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:03 2022] DELETE /identity/v3/projects/fff526044867472b8a863606aeabaf21 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:04.278915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.279490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.285272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.285850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.286135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.286408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] PATH_INFO: `/v3/users/b37556f1507a4b37bf5ce128a38090dc` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.289283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37556f1507a4b37bf5ce128a38090dc)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.293031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.306334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfc4d1-3203-4b9f-a12a-c3cf423c00df admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.318016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317a302-1183-4fa5-9f55-89db50023b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.325999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a302-1183-4fa5-9f55-89db50023b90 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.325999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a302-1183-4fa5-9f55-89db50023b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.325999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317a302-1183-4fa5-9f55-89db50023b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.325999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317a302-1183-4fa5-9f55-89db50023b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.325999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e317a302-1183-4fa5-9f55-89db50023b90 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.348003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 972/1931] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:04.360329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.360329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.360329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.381128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:04.391989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.399481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 973/1932] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:04.399481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.413158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:04.417928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7fe9e8-62a3-40d8-be72-c73ec9b861c4 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.424829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e95c5-0931-486d-a82b-b9c3fecf3b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.425737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e95c5-0931-486d-a82b-b9c3fecf3b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.426153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e95c5-0931-486d-a82b-b9c3fecf3b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.426525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e95c5-0931-486d-a82b-b9c3fecf3b49 admin admin] PATH_INFO: `/v3/projects/1f0442a3f55049858c7e9d7d927f5311/users/6c3c691b884f42d3916a7d95d93c7321/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.447035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e95c5-0931-486d-a82b-b9c3fecf3b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0442a3f55049858c7e9d7d927f5311, user_id=6c3c691b884f42d3916a7d95d93c7321, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.449857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e95c5-0931-486d-a82b-b9c3fecf3b49 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.525893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37556f1507a4b37bf5ce128a38090dc'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.540613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37556f1507a4b37bf5ce128a38090dc'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.541487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37556f1507a4b37bf5ce128a38090dc'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.549487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37556f1507a4b37bf5ce128a38090dc'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.565549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 974/1933] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:04 2022] PUT /identity/v3/projects/1f0442a3f55049858c7e9d7d927f5311/users/6c3c691b884f42d3916a7d95d93c7321/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:04.566715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.567125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.567558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.598137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:04.599855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.627655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37556f1507a4b37bf5ce128a38090dc'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:04.638010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 960/1934] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:04 2022] DELETE /identity/v3/users/b37556f1507a4b37bf5ce128a38090dc => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:04.639351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.652748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.699931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 975/1935] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:04.705456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.718793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db443fc-6364-4561-ad84-fc005bd1847b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.748059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bffd0-8021-4fd1-8a3a-f666e12c58cd admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.754647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.755303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.755647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.756484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.757505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.759474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.771675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 961/1936] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:04.775644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.786996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f8cfb-9cd2-4635-8056-06c442ae9bfc admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.789524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.790243 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.790617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.790959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.791707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:04.792075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.792167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27389adf-beb4-4571-b139-8967bc6d4fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.792790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27389adf-beb4-4571-b139-8967bc6d4fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.793067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27389adf-beb4-4571-b139-8967bc6d4fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.793335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27389adf-beb4-4571-b139-8967bc6d4fa0 admin admin] PATH_INFO: `/v3/projects/1f0442a3f55049858c7e9d7d927f5311/users/6c3c691b884f42d3916a7d95d93c7321/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.794059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.795384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27389adf-beb4-4571-b139-8967bc6d4fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f0442a3f55049858c7e9d7d927f5311, user_id=6c3c691b884f42d3916a7d95d93c7321, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.799076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27389adf-beb4-4571-b139-8967bc6d4fa0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.807366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 976/1937] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:04.813504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 962/1938] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:04 2022] PUT /identity/v3/projects/1f0442a3f55049858c7e9d7d927f5311/users/6c3c691b884f42d3916a7d95d93c7321/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:04.814547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.823729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.824162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.824995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.825093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933068-ad4c-4501-b1ff-153640d87d01 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:04.857263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] MFA Rules not processed for user `6c3c691b884f42d3916a7d95d93c7321`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:04.859299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.914370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:04.915336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:04.915742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:04.916304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:04.917538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:04.919124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:04.945882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 963/1939] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:04.957340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:04.959043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:04.993126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 977/1940] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:04 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:04.996924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.036478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4ba85-a130-403d-9089-4c42d5b7693a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.072427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66b710-7d26-4688-bce8-b8815ddd9cb8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.077906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.078586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.078751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.078991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.079668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.081575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.093853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.094514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.094749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.095006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.133167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:05.174803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.176042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.181528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 978/1941] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:04 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:05.181528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.182385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:05.187692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 964/1942] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:04 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:05.188356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.198478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10b5f6-9470-4ddc-a7b7-5408b915815e service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.204614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.205243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.205540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.205788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.206458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.209190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.212356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd3c07-a667-4703-af85-5cbd531e5b92 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.222895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 965/1943] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:05.223774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.224840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.225540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.226173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] PATH_INFO: `/v3/projects/5360336ba40f4d77a585b5c79f1bddc8` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.226631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.228506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5360336ba40f4d77a585b5c79f1bddc8)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.231504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.242728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07464d1-dcef-4ce3-ad65-1bb76559c2c8 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.247860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.248651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.252566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.252566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.258142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.260331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.273038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 966/1944] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:05.274101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.279246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:05.293315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe09e6-e5e5-438f-afd7-4580cbfa5667 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.297728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.298447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.298791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.299135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.301221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:05.374624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.375778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.386590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 967/1945] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:05.387507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.424499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5360336ba40f4d77a585b5c79f1bddc8', 'request_id': 'req-50f63a60-52b6-4985-ab3d-272d6d3b2150'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:05.428405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae81558-4709-4f91-a5a0-fd6efb978d57 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.439988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5360336ba40f4d77a585b5c79f1bddc8', 'request_id': 'req-50f63a60-52b6-4985-ab3d-272d6d3b2150'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:05.455532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5360336ba40f4d77a585b5c79f1bddc8', 'request_id': 'req-50f63a60-52b6-4985-ab3d-272d6d3b2150'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:05.527636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae477518-8f08-436b-b3f0-835d989e16a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.528311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae477518-8f08-436b-b3f0-835d989e16a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.528586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae477518-8f08-436b-b3f0-835d989e16a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.528856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae477518-8f08-436b-b3f0-835d989e16a4 admin admin] PATH_INFO: `/v3/projects/ab5c861a64844b068e8798410fa5054a/users/648b19a74b394e7183b399fea981907e/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.544772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae477518-8f08-436b-b3f0-835d989e16a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5c861a64844b068e8798410fa5054a, user_id=648b19a74b394e7183b399fea981907e, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.547977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae477518-8f08-436b-b3f0-835d989e16a4 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.550934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] The token cache is being invalidate because project 5360336ba40f4d77a585b5c79f1bddc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:05.550934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:05.551548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-50f63a60-52b6-4985-ab3d-272d6d3b2150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:05.553232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 979/1946] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:05 2022] DELETE /identity/v3/projects/5360336ba40f4d77a585b5c79f1bddc8 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:05.557977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d155-8f72-4376-968a-4cbc13232070 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.558263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d155-8f72-4376-968a-4cbc13232070 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.558541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d155-8f72-4376-968a-4cbc13232070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.578398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8724d155-8f72-4376-968a-4cbc13232070 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:05.579698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8724d155-8f72-4376-968a-4cbc13232070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:05.588245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 980/1947] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:05.597282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.597672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.597989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.619209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:05.620496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:05.631804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 981/1948] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:05.633999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 968/1949] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:05 2022] PUT /identity/v3/projects/ab5c861a64844b068e8798410fa5054a/users/648b19a74b394e7183b399fea981907e/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:05.641373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.649312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.649628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.650068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.652562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfe872-38a1-4724-835c-8d2259565399 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.697091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] MFA Rules not processed for user `648b19a74b394e7183b399fea981907e`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:05.697091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:05.729020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.729744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.730087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.730426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.731138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:05.731508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.733104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.746542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 982/1950] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:05.756533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.765370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 969/1951] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:05.774887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c041c1-cb4b-4a76-9097-39a330efcbbf admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.781987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:05.781987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:05.787982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:05.787982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:05.787982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:05.787982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:05.943738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 983/1952] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:05 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:05.948527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:05.959601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0cf2d-30da-47ec-b4ad-6a5f3d2372f4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:05.967470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:06.025078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.025721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.026001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.026274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.026897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.028864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.154478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 970/1953] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:05 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:06.159182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.173246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9423de-a030-4ba4-bb14-425c1bc016b5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:06.178566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.179337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.179684 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.180024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.180748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.182531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.194628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 984/1954] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:06.198482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.209113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb9e92-fee0-412d-a2fb-dbd4ad26a85d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:06.213874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c913e04-a7bd-4919-bbc9-3158b14f1917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.214486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c913e04-a7bd-4919-bbc9-3158b14f1917 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.214764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c913e04-a7bd-4919-bbc9-3158b14f1917 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.215035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c913e04-a7bd-4919-bbc9-3158b14f1917 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.215726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c913e04-a7bd-4919-bbc9-3158b14f1917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.217384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c913e04-a7bd-4919-bbc9-3158b14f1917 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.228489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 971/1955] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:06.232464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.244303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e625e6-3f7f-4efc-9b05-fab4638b2727 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:06.249401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3828d-82a0-4872-96ad-cf8a6ffce686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.250534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3828d-82a0-4872-96ad-cf8a6ffce686 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.252836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e4b5-97a0-4fa0-9900-ee4dbba987a6 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.253124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e4b5-97a0-4fa0-9900-ee4dbba987a6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.253398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880e4b5-97a0-4fa0-9900-ee4dbba987a6 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.254959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3828d-82a0-4872-96ad-cf8a6ffce686 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.255657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 972/1956] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:33:06.256088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3828d-82a0-4872-96ad-cf8a6ffce686 admin admin] PATH_INFO: `/v3/projects/fbabc8cd2aa34a389122aa28029517bd/users/6dd83966138b46eba53a98e20eb1f26e/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.278037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3828d-82a0-4872-96ad-cf8a6ffce686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbabc8cd2aa34a389122aa28029517bd, user_id=6dd83966138b46eba53a98e20eb1f26e, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.284151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e4b5-97a0-4fa0-9900-ee4dbba987a6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.285994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2880e4b5-97a0-4fa0-9900-ee4dbba987a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:06.292581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3828d-82a0-4872-96ad-cf8a6ffce686 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.377333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880e4b5-97a0-4fa0-9900-ee4dbba987a6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:06.394133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3011e1-42fc-4005-bca5-bbc23493892d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.395217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3011e1-42fc-4005-bca5-bbc23493892d service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.395356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3011e1-42fc-4005-bca5-bbc23493892d service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.398193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3011e1-42fc-4005-bca5-bbc23493892d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.423069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 985/1957] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:06 2022] PUT /identity/v3/projects/fbabc8cd2aa34a389122aa28029517bd/users/6dd83966138b46eba53a98e20eb1f26e/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:06.433336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.433624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.433942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.489957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:06.494851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:06.498042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3011e1-42fc-4005-bca5-bbc23493892d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.501558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3011e1-42fc-4005-bca5-bbc23493892d service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.517658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 973/1958] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:06.519475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.519764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.520035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.560527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] MFA Rules not processed for user `6dd83966138b46eba53a98e20eb1f26e`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:06.563996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:06.682725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 974/1959] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:06.689210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.755115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 986/1960] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:33:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:06.772171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.774767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:06.797995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6e23e-d7e7-434a-8923-d1dd21c2368d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:06.814024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dfc83-b1a4-49e0-a1d8-eecce28c5e31 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:06.823932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.823932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.823932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.823932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.824359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:06.825011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:06.825291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:06.825573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:06.826185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.828776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.830817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:06.832624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:06.845826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 987/1961] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:06.951564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 975/1962] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:06 2022] POST /identity/v3/projects => generated 385 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:06.955908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:06.966997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972cab8-1b02-4f48-ac1e-1d1e08c77a74 service nova] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:07.028725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:07.029366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.029643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.029915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.030701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:07.032649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:07.164318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 988/1963] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:06 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:07.167939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:07.179919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68630574-e02b-4abf-a68b-a05fa1f04409 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:07.184581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:07.185183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.185487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.185759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.186438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:07.188342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:07.198445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 976/1964] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:07.203252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:07.213452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479694d2-77c7-42e0-bfce-e452070d4707 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:07.219202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:07.219886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.220165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.227831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] PATH_INFO: `/v3/projects/991972a8521048048a2227e5122d6dd0/users/59da44c8e4fb46d38f8cd5a5d0d92b68/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.236895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991972a8521048048a2227e5122d6dd0, user_id=59da44c8e4fb46d38f8cd5a5d0d92b68, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:07.239609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:07.328401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 989/1965] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:07 2022] PUT /identity/v3/projects/991972a8521048048a2227e5122d6dd0/users/59da44c8e4fb46d38f8cd5a5d0d92b68/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:07.330878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:07.342244 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa88deea-ea7e-4bc2-ae64-84f1aef97aab admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:07.414861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2822a00-9795-4b2e-880a-f1bb80985ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:07.416625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2822a00-9795-4b2e-880a-f1bb80985ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.417107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2822a00-9795-4b2e-880a-f1bb80985ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.417887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2822a00-9795-4b2e-880a-f1bb80985ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.431793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2822a00-9795-4b2e-880a-f1bb80985ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:07.431793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2822a00-9795-4b2e-880a-f1bb80985ca7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:07.442316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 977/1966] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:07.443150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:07.459166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84acdb60-f88f-4171-ab9c-76489c990b92 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:07.466130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:07.467217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.468300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.468490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] PATH_INFO: `/v3/projects/991972a8521048048a2227e5122d6dd0/users/59da44c8e4fb46d38f8cd5a5d0d92b68/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.470705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] RBAC: Authorizing `identity:create_grant(project_id=991972a8521048048a2227e5122d6dd0, user_id=59da44c8e4fb46d38f8cd5a5d0d92b68, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:07.473505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:07.496922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.497331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.497682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.500875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 990/1967] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:07 2022] PUT /identity/v3/projects/991972a8521048048a2227e5122d6dd0/users/59da44c8e4fb46d38f8cd5a5d0d92b68/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:07.540840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] MFA Rules not processed for user `59da44c8e4fb46d38f8cd5a5d0d92b68`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:07.542431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:07.571426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:07.640402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 978/1968] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:07.653390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:07.755563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:07.755563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:07.821320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5e57-a8f3-4f5b-85e2-225c9c59436a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:07.825269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5fa55f-2c41-47dd-aef6-e99c8b66d22c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:07.825875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fa55f-2c41-47dd-aef6-e99c8b66d22c service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:07.826154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fa55f-2c41-47dd-aef6-e99c8b66d22c service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:07.826425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5fa55f-2c41-47dd-aef6-e99c8b66d22c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:07.885375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fa55f-2c41-47dd-aef6-e99c8b66d22c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:07.886818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5fa55f-2c41-47dd-aef6-e99c8b66d22c service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:07.896228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 991/1969] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:09.274503 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:09.351191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:10.024259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:10.783409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:11.303922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:11.685028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:12.041790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:12.241580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:12.459836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:12.876797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:13.930344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:14.002066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:14.492533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:14.493940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:14.558118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558677c-27a9-4551-8aa5-302fb77a07f5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:14.564238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:14.564948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:14.565232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:14.565503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] PATH_INFO: `/v3/users/4022f60ce6db4f3a9419bc4ed16ba643` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:14.566668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4022f60ce6db4f3a9419bc4ed16ba643)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:14.568717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:14.721296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4022f60ce6db4f3a9419bc4ed16ba643'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:14.733751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4022f60ce6db4f3a9419bc4ed16ba643'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:14.734547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4022f60ce6db4f3a9419bc4ed16ba643'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:14.741412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4022f60ce6db4f3a9419bc4ed16ba643'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:14.827108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4022f60ce6db4f3a9419bc4ed16ba643'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:14.833634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 979/1970] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:14 2022] DELETE /identity/v3/users/4022f60ce6db4f3a9419bc4ed16ba643 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:14.838391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:14.838705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:14.839013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:14.859010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:14.864827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:14.909870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:14.921839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 992/1971] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:14.931568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:14.932954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:14.960421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:14.991960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e753a-10f3-489f-b0d2-733c83527c93 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:14.995616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:14.996740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:14.996740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:14.996740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:15.005477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:15.006889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:15.016113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 980/1972] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:15.432514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:15.434629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:15.446707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e28cd-9e81-4d9c-a310-2e923fdcb980 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:15.452286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:15.452993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:15.453563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:15.454019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] PATH_INFO: `/v3/projects/30dca8d1c34242aa8b4c4fe638d85a53` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:15.455217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] RBAC: Authorizing `identity:delete_project(project_id=30dca8d1c34242aa8b4c4fe638d85a53)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:15.457298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:15.486387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:15.579250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30dca8d1c34242aa8b4c4fe638d85a53', 'request_id': 'req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.584908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30dca8d1c34242aa8b4c4fe638d85a53', 'request_id': 'req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.596459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30dca8d1c34242aa8b4c4fe638d85a53', 'request_id': 'req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.663983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] The token cache is being invalidate because project 30dca8d1c34242aa8b4c4fe638d85a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:15.664277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.664905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.666925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 993/1973] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:15 2022] DELETE /identity/v3/projects/30dca8d1c34242aa8b4c4fe638d85a53 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:15.671475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:15.681146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab087034-ecfe-4c90-b4f3-1a7fa25287a2 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:15.690307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:15.737771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:15.738405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:15.738683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:15.738955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] PATH_INFO: `/v3/users/6c3c691b884f42d3916a7d95d93c7321` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:15.740105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c3c691b884f42d3916a7d95d93c7321)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:15.742028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:15.905398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:15.906586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c3c691b884f42d3916a7d95d93c7321'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.918718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c3c691b884f42d3916a7d95d93c7321'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.919508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c3c691b884f42d3916a7d95d93c7321'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.927283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c3c691b884f42d3916a7d95d93c7321'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.958727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:15.981077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ebcd5186-a35e-47a7-9a13-03f3d2a21657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c3c691b884f42d3916a7d95d93c7321'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:15.988030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 981/1974] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:15 2022] DELETE /identity/v3/users/6c3c691b884f42d3916a7d95d93c7321 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:16.264397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:16.321376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:16.386982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:16.388724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:16.402911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbb5fa-3ba0-48cf-8cc5-ac677480983a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:16.461115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:16.461760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:16.462045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:16.462321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] PATH_INFO: `/v3/projects/1f0442a3f55049858c7e9d7d927f5311` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:16.463614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f0442a3f55049858c7e9d7d927f5311)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:16.465536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:16.565574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f0442a3f55049858c7e9d7d927f5311', 'request_id': 'req-a9d156e3-839a-49a2-8cc5-553b6e9951d1'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:16.571307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f0442a3f55049858c7e9d7d927f5311', 'request_id': 'req-a9d156e3-839a-49a2-8cc5-553b6e9951d1'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:16.577703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f0442a3f55049858c7e9d7d927f5311', 'request_id': 'req-a9d156e3-839a-49a2-8cc5-553b6e9951d1'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:16.659661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] The token cache is being invalidate because project 1f0442a3f55049858c7e9d7d927f5311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:16.659957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:16.661062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a9d156e3-839a-49a2-8cc5-553b6e9951d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:16.662117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 994/1975] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:16 2022] DELETE /identity/v3/projects/1f0442a3f55049858c7e9d7d927f5311 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:16.666967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:16.667287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:16.667568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:16.687407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:16.688614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:16.748246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 982/1976] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:16.756139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:16.756430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:16.756778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:16.765568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:16.777619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:16.778848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:16.787961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 995/1977] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:16.792295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:16.802397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42e3bc-19a4-47d1-9b5f-eb26a92efd8d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:16.806976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:16.807670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:16.807947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:16.808215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:16.808895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:16.809194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:16.810769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:16.822291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 983/1978] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:16.826582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:16.836850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0fac5-295c-4cd7-993d-8c98809b3608 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:16.841750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:16.842446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:16.843461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:16.843763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:16.844407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:16.846308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:16.901607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:17.003523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 996/1979] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:16 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:17.004855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.015674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633194f-4b4d-452b-9e0a-8e7ab72464b9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.074236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.074969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.075275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.075583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.076226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.078699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.206894 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 984/1980] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:17 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:17.211088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.227721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f3081-547c-4932-80a0-d217067f6a35 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.232710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.233399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.234638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.234638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.234638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.236365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.247321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 997/1981] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:17.252573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.263727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1c19cb-0228-417a-9ed2-2446f2cd1e38 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.269602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.270265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.270483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.270752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.271431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.273066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.289938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 985/1982] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:17.294792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.301644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad84a3-f58d-4b57-9efc-262881e8bd42 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.306424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b8308-47de-474b-a6ed-d7a7effce741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.307147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b8308-47de-474b-a6ed-d7a7effce741 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.307380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b8308-47de-474b-a6ed-d7a7effce741 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.307650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b8308-47de-474b-a6ed-d7a7effce741 admin admin] PATH_INFO: `/v3/projects/3a95e10bd5074bb3be559a37437c1700/users/5527cfe0264149089f2ce80224c9d741/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.329180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b8308-47de-474b-a6ed-d7a7effce741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a95e10bd5074bb3be559a37437c1700, user_id=5527cfe0264149089f2ce80224c9d741, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.329309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:17.329416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b8308-47de-474b-a6ed-d7a7effce741 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.329870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.331584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:17.387742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed7639-8c4f-4c11-9669-610350bf0f94 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.393091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.393724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.393983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.394243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.396182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:17.405769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 998/1983] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:17 2022] PUT /identity/v3/projects/3a95e10bd5074bb3be559a37437c1700/users/5527cfe0264149089f2ce80224c9d741/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:17.408854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:17.410162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.410516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.410856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.440621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] MFA Rules not processed for user `5527cfe0264149089f2ce80224c9d741`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:17.442033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:17.462956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.464478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.479848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 986/1984] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:17.515107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 999/1985] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:17.519612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.530787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febcf91-3f72-4f6c-bd09-a7071ae81153 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.643301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.645975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.645975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.645975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.645975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.647196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.774365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 987/1986] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:17 2022] POST /identity/v3/projects => generated 401 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:17.778816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.792273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4743b612-c561-44fa-9b14-2e2a091600eb None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.854867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.855507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.855782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.856050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:17.856677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:17.859457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:17.860282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:17.972104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:17.978138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1000/1987] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:17 2022] POST /identity/v3/users => generated 307 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:17.981930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:17.993310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4fd15-2c4d-4b96-b5a1-a9460f945ff5 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:17.998033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:17.998848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:17.999141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:17.999490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:18.000078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:18.001719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:18.015928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 988/1988] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:18.017185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:18.027611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75db3c5-0c9c-4f34-90aa-1224ba900fc0 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:18.033054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:18.033873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:18.034056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:18.034355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] PATH_INFO: `/v3/projects/cc99db8fa2bf4a61958c84f1331aa81a/users/06d953e35f2f4ebfa9700ba6f44196cf/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:18.050425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc99db8fa2bf4a61958c84f1331aa81a, user_id=06d953e35f2f4ebfa9700ba6f44196cf, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:18.053162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:18.133287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1001/1989] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:18 2022] PUT /identity/v3/projects/cc99db8fa2bf4a61958c84f1331aa81a/users/06d953e35f2f4ebfa9700ba6f44196cf/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:18.137268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:18.149107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2ca26-6f7a-44d1-909b-f1d8f72aceaa admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:18.207424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590a954-2bb1-4c35-8683-e0216a83d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:18.208113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590a954-2bb1-4c35-8683-e0216a83d266 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:18.208477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590a954-2bb1-4c35-8683-e0216a83d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:18.208684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590a954-2bb1-4c35-8683-e0216a83d266 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:18.209596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590a954-2bb1-4c35-8683-e0216a83d266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:18.215364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590a954-2bb1-4c35-8683-e0216a83d266 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:18.228207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 989/1990] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:18.233726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:18.266375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e93329-c220-40a5-ad63-04007a78e982 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:18.272018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:18.272915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:18.273405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:18.274467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] PATH_INFO: `/v3/projects/cc99db8fa2bf4a61958c84f1331aa81a/users/06d953e35f2f4ebfa9700ba6f44196cf/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:18.283968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc99db8fa2bf4a61958c84f1331aa81a, user_id=06d953e35f2f4ebfa9700ba6f44196cf, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:18.287808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:18.301546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1002/1991] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:18 2022] PUT /identity/v3/projects/cc99db8fa2bf4a61958c84f1331aa81a/users/06d953e35f2f4ebfa9700ba6f44196cf/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:18.312941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:18.313426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:18.320418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:18.347822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] MFA Rules not processed for user `06d953e35f2f4ebfa9700ba6f44196cf`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:18.349390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:18.442316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 990/1992] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:18 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:18.489338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:18.524566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:18.526288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:18.585947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca1ca3-dd87-40b5-a388-39811e592f96 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:18.589715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:18.590350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:18.590634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:18.590988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:18.600721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:18.602145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:18.612389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1003/1993] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:18 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:18.847909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:19.090407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:19.129365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:19.138884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da3482-aea3-4d2e-b248-e94e74a53056 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:19.143368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01766634-2567-4fb2-b6f5-94f966d62316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:19.143973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01766634-2567-4fb2-b6f5-94f966d62316 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:19.144253 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01766634-2567-4fb2-b6f5-94f966d62316 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:19.144522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01766634-2567-4fb2-b6f5-94f966d62316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:19.207316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01766634-2567-4fb2-b6f5-94f966d62316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:19.208791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01766634-2567-4fb2-b6f5-94f966d62316 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:19.217929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 991/1994] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:19.477114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:19.527545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.164617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.230222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.435340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.470451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.564158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=80,keystone:DELETE=16,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.586993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.827565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:20.910228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.052819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.075586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.121195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.144313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.212935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.281284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=2,keystone:INSERT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.307123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.370350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.766921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:21.966922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:28.395265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:28.396708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:28.478176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0c4ae-9a81-4f66-b89e-b4c4af8c4de2 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:28.483289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:28.483933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:28.484245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:28.484536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] PATH_INFO: `/v3/users/8e7a76bc1ec0409399318f8b609db224` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:28.485718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7a76bc1ec0409399318f8b609db224)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:28.488342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:28.704301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7a76bc1ec0409399318f8b609db224'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:28.716313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:28.718099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7a76bc1ec0409399318f8b609db224'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:28.718885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7a76bc1ec0409399318f8b609db224'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:28.729359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7a76bc1ec0409399318f8b609db224'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:28.815756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7a76bc1ec0409399318f8b609db224'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:28.822738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1004/1995] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:28 2022] DELETE /identity/v3/users/8e7a76bc1ec0409399318f8b609db224 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:28.827037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:28.827526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:28.827740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:28.847812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:28.849041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:28.900415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:28.909398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 992/1996] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:28.919949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:28.921339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:28.987588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd482fb7-55c1-43fc-b5cc-cc204b84aec9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:28.992462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:28.992937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:28.993261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:28.993609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:29.006857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:29.008334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:29.020099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1005/1997] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:29.434155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:29.438528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:29.448014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad542ce-d7e6-485e-9dcd-83b9060aa00f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:29.453133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:29.453760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:29.455558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:29.455843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] PATH_INFO: `/v3/projects/f3b7101185d64cdd9028082404030a03` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:29.457048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b7101185d64cdd9028082404030a03)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:29.458940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:29.568841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b7101185d64cdd9028082404030a03', 'request_id': 'req-534fd751-2488-4fd2-869e-6f9c49821362'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.574927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b7101185d64cdd9028082404030a03', 'request_id': 'req-534fd751-2488-4fd2-869e-6f9c49821362'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.583946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b7101185d64cdd9028082404030a03', 'request_id': 'req-534fd751-2488-4fd2-869e-6f9c49821362'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.661917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] The token cache is being invalidate because project f3b7101185d64cdd9028082404030a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:29.662206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.662844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.664377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 993/1998] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:29 2022] DELETE /identity/v3/projects/f3b7101185d64cdd9028082404030a03 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:29.668350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:29.678510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a57b93-56b6-4ff5-ac75-b5dccae7fe09 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:29.738364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:29.739018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:29.739334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:29.739607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] PATH_INFO: `/v3/users/e98db5c482ea49d6b2369b14a581088e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:29.740724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98db5c482ea49d6b2369b14a581088e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:29.742689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:29.856494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:29.858155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:29.873498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534fd751-2488-4fd2-869e-6f9c49821362 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:29.878754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:29.879423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:29.879732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:29.880106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] PATH_INFO: `/v3/users/6dd83966138b46eba53a98e20eb1f26e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:29.881264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd83966138b46eba53a98e20eb1f26e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:29.883215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:29.925808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98db5c482ea49d6b2369b14a581088e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.940594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98db5c482ea49d6b2369b14a581088e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.941290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98db5c482ea49d6b2369b14a581088e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:29.948228 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98db5c482ea49d6b2369b14a581088e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.033230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-377d28e9-359c-4533-9b81-184fc5ea60c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98db5c482ea49d6b2369b14a581088e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.044283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1006/1999] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:29 2022] DELETE /identity/v3/users/e98db5c482ea49d6b2369b14a581088e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:30.131607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd83966138b46eba53a98e20eb1f26e'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.148656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd83966138b46eba53a98e20eb1f26e'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.149488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd83966138b46eba53a98e20eb1f26e'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.158447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd83966138b46eba53a98e20eb1f26e'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.272727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd83966138b46eba53a98e20eb1f26e'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.277807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 994/2000] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:29 2022] DELETE /identity/v3/users/6dd83966138b46eba53a98e20eb1f26e => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:30.282084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.282372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.282680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.290490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:30.304194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:30.305427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.372255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1007/2001] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:30.382591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:30.384144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.420251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:30.421685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.441308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a4e33-63fa-4695-aec0-a09e2528076c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:30.453876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:30.454567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.454851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.455128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] PATH_INFO: `/v3/users/5527cfe0264149089f2ce80224c9d741` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.458161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5527cfe0264149089f2ce80224c9d741)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:30.460466 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:30.468022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894acce-b6a4-4748-b423-cbb7c07e5cd1 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:30.473568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:30.474266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.474548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.474970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.489566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:30.491018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:30.495269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:30.501914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 995/2002] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:30.552260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:30.553951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.569475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384933f-2a4d-4201-95a1-636345468bb4 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:30.577468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:30.577468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.577468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.577468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] PATH_INFO: `/v3/projects/a1d1b2aedf174fd7b0ddd2f974d1f007` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.577872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d1b2aedf174fd7b0ddd2f974d1f007)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:30.579108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:30.695445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5527cfe0264149089f2ce80224c9d741'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.696579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d1b2aedf174fd7b0ddd2f974d1f007', 'request_id': 'req-f61d859e-def7-4d75-83ec-7c2649195255'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.707719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d1b2aedf174fd7b0ddd2f974d1f007', 'request_id': 'req-f61d859e-def7-4d75-83ec-7c2649195255'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.711152 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5527cfe0264149089f2ce80224c9d741'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.712029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5527cfe0264149089f2ce80224c9d741'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.716702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d1b2aedf174fd7b0ddd2f974d1f007', 'request_id': 'req-f61d859e-def7-4d75-83ec-7c2649195255'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.722237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5527cfe0264149089f2ce80224c9d741'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.802002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] The token cache is being invalidate because project a1d1b2aedf174fd7b0ddd2f974d1f007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:30.802343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.803169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f61d859e-def7-4d75-83ec-7c2649195255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.804812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 996/2003] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:30 2022] DELETE /identity/v3/projects/a1d1b2aedf174fd7b0ddd2f974d1f007 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:30.810269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac2e6-ae15-4707-a53c-a0bece6cbb0c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.810572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac2e6-ae15-4707-a53c-a0bece6cbb0c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.810854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac2e6-ae15-4707-a53c-a0bece6cbb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.831831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f7dac2e6-ae15-4707-a53c-a0bece6cbb0c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:30.833047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f7dac2e6-ae15-4707-a53c-a0bece6cbb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.888203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-69bdc37d-0c29-4c58-a0dd-c38ce5e2acbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5527cfe0264149089f2ce80224c9d741'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:30.902204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1008/2004] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:30 2022] DELETE /identity/v3/users/5527cfe0264149089f2ce80224c9d741 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:30.902307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 997/2005] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:30.902730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.902730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.902730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.905460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:30.905460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:30.905654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:30.921632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:30.922930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.925964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:30.927144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:30.995072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1009/2006] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:30.995972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:30.997235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.004621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 998/2007] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:31.005663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.006982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.013963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c464079-73f1-4997-9f69-c743da11d84c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.018821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.019491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.019778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.020286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] PATH_INFO: `/v3/projects/fbabc8cd2aa34a389122aa28029517bd` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.021235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbabc8cd2aa34a389122aa28029517bd)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.023143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.034946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:31.066439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf182481-c3f8-4f62-b18f-b25b40f34591 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.069657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.070304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.070662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.070946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.074405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:31.082853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.084661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.094399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 999/2008] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:31.095213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.104191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c27ee2-6794-48aa-bb64-a7fcafd70e3f service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.109014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.109629 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.109903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.110416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.111828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:31.112131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.113705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.125509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1000/2009] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:31.131245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.141637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc091e-7e3e-41ef-9f2c-46ef9f2e8948 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.155883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbabc8cd2aa34a389122aa28029517bd', 'request_id': 'req-baf45a43-4fee-4021-a21f-b703dc998abb'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.162058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbabc8cd2aa34a389122aa28029517bd', 'request_id': 'req-baf45a43-4fee-4021-a21f-b703dc998abb'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.169188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbabc8cd2aa34a389122aa28029517bd', 'request_id': 'req-baf45a43-4fee-4021-a21f-b703dc998abb'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.199397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.200010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.200290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.200560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.201180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.203442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.291484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] The token cache is being invalidate because project fbabc8cd2aa34a389122aa28029517bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:31.291898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.292695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.297123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1001/2011] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:31 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:31.297444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1010/2011] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:30 2022] DELETE /identity/v3/projects/fbabc8cd2aa34a389122aa28029517bd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:31.299520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.299650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.300783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.300895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.316591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45a43-4fee-4021-a21f-b703dc998abb admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.379358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba9cbe-d375-4261-80dc-d2bfab1abbce admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.387564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.388473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.388956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.389254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.390173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.391449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.392327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.392491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.393075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.393548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] PATH_INFO: `/v3/users/59da44c8e4fb46d38f8cd5a5d0d92b68` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.395188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59da44c8e4fb46d38f8cd5a5d0d92b68)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.398120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.494113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1002/2012] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:31 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:31.498075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.508303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18ab8a-4843-4e1a-9d24-90d291e4fd87 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.513591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.516813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.516813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.516813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.516813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.517167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.527042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1003/2013] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:31.530823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.541056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6d09e-1295-4552-93ce-0b585fbba803 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.545815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.546407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.546684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.557276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.557276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.557276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.559855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1004/2014] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:31.563656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.573983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab890db-2898-4e5a-9ef1-0341e1674fcd admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.579003 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.580894 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.581208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.581487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] PATH_INFO: `/v3/projects/f1b7329e2ca84e198d1c1e5869286a9f/users/4aca6f188d2f40e4a816c0c148fdbef0/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.603543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b7329e2ca84e198d1c1e5869286a9f, user_id=4aca6f188d2f40e4a816c0c148fdbef0, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.607095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.621960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59da44c8e4fb46d38f8cd5a5d0d92b68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.634882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59da44c8e4fb46d38f8cd5a5d0d92b68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.635594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59da44c8e4fb46d38f8cd5a5d0d92b68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.642428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59da44c8e4fb46d38f8cd5a5d0d92b68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.718733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-94e0bfd6-97cd-4672-996e-e12918e8db4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59da44c8e4fb46d38f8cd5a5d0d92b68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.725036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1005/2015] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:31 2022] PUT /identity/v3/projects/f1b7329e2ca84e198d1c1e5869286a9f/users/4aca6f188d2f40e4a816c0c148fdbef0/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:31.725519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1011/2016] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:31 2022] DELETE /identity/v3/users/59da44c8e4fb46d38f8cd5a5d0d92b68 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:31.725903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.727549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.734944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:31.737877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.738672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.738954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.768535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] MFA Rules not processed for user `4aca6f188d2f40e4a816c0c148fdbef0`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:31.769954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.776200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:31.807332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f25290-1565-4921-8f7f-88ff74b28e50 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.842781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1012/2017] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:31.848006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:31.849386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:31.871073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.871810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.872092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.872365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] PATH_INFO: `/v3/projects/3a95e10bd5074bb3be559a37437c1700` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.873538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a95e10bd5074bb3be559a37437c1700)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.875477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.916984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10147fe8-804e-48db-8b71-f674b60e308b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:31.919478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:31.920095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:31.920376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:31.920646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:31.921270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:31.923722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:31.986741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a95e10bd5074bb3be559a37437c1700', 'request_id': 'req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:31.993510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a95e10bd5074bb3be559a37437c1700', 'request_id': 'req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.000141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a95e10bd5074bb3be559a37437c1700', 'request_id': 'req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.041670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1013/2018] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:31 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:32.048050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.068063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] The token cache is being invalidate because project 3a95e10bd5074bb3be559a37437c1700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:32.068353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.069004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.070761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1006/2019] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:31 2022] DELETE /identity/v3/projects/3a95e10bd5074bb3be559a37437c1700 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:32.074379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.075737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:32.115635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16f7d2-f26c-4035-9d9d-5ad76f374690 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.121028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.121642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.122307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.122307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.122869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.125315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.147881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:32.151580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f696eaf-4ae1-4ed1-a13c-71e35cc71811 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.156450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.157081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.157361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.158359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] PATH_INFO: `/v3/users/06d953e35f2f4ebfa9700ba6f44196cf` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.159883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d953e35f2f4ebfa9700ba6f44196cf)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.161815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.224224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1014/2020] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:32 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:32.224224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.224224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:32.234291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dba3b-982b-4164-a80f-f2592371294b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.240351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.241009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.241319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.241593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] PATH_INFO: `/v3/projects/991972a8521048048a2227e5122d6dd0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.242886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=991972a8521048048a2227e5122d6dd0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.244948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.423286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d953e35f2f4ebfa9700ba6f44196cf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.424279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991972a8521048048a2227e5122d6dd0', 'request_id': 'req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.429850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991972a8521048048a2227e5122d6dd0', 'request_id': 'req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.435159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d953e35f2f4ebfa9700ba6f44196cf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.435862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d953e35f2f4ebfa9700ba6f44196cf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.436428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991972a8521048048a2227e5122d6dd0', 'request_id': 'req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.442890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d953e35f2f4ebfa9700ba6f44196cf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.526699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] The token cache is being invalidate because project 991972a8521048048a2227e5122d6dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:32.526989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.527661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.534429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1015/2021] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:32 2022] DELETE /identity/v3/projects/991972a8521048048a2227e5122d6dd0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:32.534429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.534429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:32.535602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:32.599724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed7f32-bc2d-4eb0-94ae-c46cb3fe2451 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.604513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.605129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.605408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.605677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.606314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.607604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2dee9ccd-5ac3-45ce-980a-c1269840bc02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d953e35f2f4ebfa9700ba6f44196cf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:32.608086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.618340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1007/2022] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:32 2022] DELETE /identity/v3/users/06d953e35f2f4ebfa9700ba6f44196cf => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:32.620506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f425c-dd10-4090-9026-1e948cc3b14d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.620870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f425c-dd10-4090-9026-1e948cc3b14d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.621238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f425c-dd10-4090-9026-1e948cc3b14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.622686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1016/2023] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:32.623822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.625820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:32.640656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf69b5-8431-4a64-9fba-961386022957 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.651587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ad8f425c-dd10-4090-9026-1e948cc3b14d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:32.651587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ad8f425c-dd10-4090-9026-1e948cc3b14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:32.707970 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.709128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.709128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.709698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] PATH_INFO: `/v3/projects/530b92c81ee249f0884014abbcb50752/users/8cb878467ce44140a43e8846336876fa/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.726174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530b92c81ee249f0884014abbcb50752, user_id=8cb878467ce44140a43e8846336876fa, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.729236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.729910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1008/2024] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:32.737725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.738018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.738291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.757659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:32.758861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:32.767721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1009/2025] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:32.771588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.781051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd9599-7478-4a8c-9565-5eb57c6b138b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.786824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.787512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.787787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.788544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.789198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:32.789495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.791108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.803417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1010/2026] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:33:32.810732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1017/2027] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:32 2022] PUT /identity/v3/projects/530b92c81ee249f0884014abbcb50752/users/8cb878467ce44140a43e8846336876fa/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:32.813549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.813672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.824228 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4e3e7-2daf-4379-9674-64706a99e1e9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.877451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e46739-aa18-4763-a7ea-ed508c18b936 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.883011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.883668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.883942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.884213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.884839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.887124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.887602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.888416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.888866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.889305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.890219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.892874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.904019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1011/2028] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:32.908576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:32.921335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bfc41-8a7e-4ee2-9580-d98e684d3bd7 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:32.928180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c203c-edb4-43ff-8b06-dfd7679ba8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:32.928180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c203c-edb4-43ff-8b06-dfd7679ba8ec admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.928180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c203c-edb4-43ff-8b06-dfd7679ba8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.928180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c203c-edb4-43ff-8b06-dfd7679ba8ec admin admin] PATH_INFO: `/v3/projects/530b92c81ee249f0884014abbcb50752/users/8cb878467ce44140a43e8846336876fa/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.930005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c203c-edb4-43ff-8b06-dfd7679ba8ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=530b92c81ee249f0884014abbcb50752, user_id=8cb878467ce44140a43e8846336876fa, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:32.932694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c203c-edb4-43ff-8b06-dfd7679ba8ec admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:32.944903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1012/2029] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:32 2022] PUT /identity/v3/projects/530b92c81ee249f0884014abbcb50752/users/8cb878467ce44140a43e8846336876fa/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:32.956615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:32.959584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:32.959584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:32.987067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] MFA Rules not processed for user `8cb878467ce44140a43e8846336876fa`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:32.988339 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.038209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1018/2030] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:32 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:33.039899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.066551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1013/2031] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:33.078425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.084989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.106841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae9766f-0b11-4936-9f5d-1bbdb1d76584 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.112603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.113223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.113631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.113913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.114629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.116532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.141951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:33.145746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172eb93-440b-4777-8ba0-53e2e3b2b845 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.196816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.197351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.197655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.197965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.199818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.223721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:33.232412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1019/2032] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:33 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:33.233454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.235240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.248209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b89fd6-b68a-4a6a-8cf7-d3d8e9b46d33 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.253944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.254624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.254912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.255181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] PATH_INFO: `/v3/projects/cc99db8fa2bf4a61958c84f1331aa81a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.256322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc99db8fa2bf4a61958c84f1331aa81a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.258318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.260373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.261882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.271165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1014/2033] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:33.271952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.282333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa42b8-ffb6-478b-b51a-2f6965fd8003 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.287499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.288152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.288436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.288711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.289353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.290983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.305200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1015/2034] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:33.306182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.318240 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e48ef2-3035-4efb-a7c1-caaef24d7666 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.323575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.324179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.324460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.324735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.325382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.326968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.341681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1016/2035] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:33.342526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.353479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4b297-2fdd-41b9-bff9-d0a1e513fe9f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.358279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6462c9-4186-482b-8300-654e31845a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.363006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6462c9-4186-482b-8300-654e31845a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.363006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6462c9-4186-482b-8300-654e31845a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.363006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6462c9-4186-482b-8300-654e31845a73 admin admin] PATH_INFO: `/v3/projects/d83e4a2162f04e55a75f21f9b65bce5a/users/90f722382aea480da062c1bec48d6b6c/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.374914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6462c9-4186-482b-8300-654e31845a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83e4a2162f04e55a75f21f9b65bce5a, user_id=90f722382aea480da062c1bec48d6b6c, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.378700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6462c9-4186-482b-8300-654e31845a73 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.397033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc99db8fa2bf4a61958c84f1331aa81a', 'request_id': 'req-abc84a58-48aa-4eb6-adb3-3044c4def6fc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.402618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc99db8fa2bf4a61958c84f1331aa81a', 'request_id': 'req-abc84a58-48aa-4eb6-adb3-3044c4def6fc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.410507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc99db8fa2bf4a61958c84f1331aa81a', 'request_id': 'req-abc84a58-48aa-4eb6-adb3-3044c4def6fc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.487917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1017/2036] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:33 2022] PUT /identity/v3/projects/d83e4a2162f04e55a75f21f9b65bce5a/users/90f722382aea480da062c1bec48d6b6c/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:33.498860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.499143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.499474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.516457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] The token cache is being invalidate because project cc99db8fa2bf4a61958c84f1331aa81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:33.517008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.518001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-abc84a58-48aa-4eb6-adb3-3044c4def6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.520103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1020/2037] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:33 2022] DELETE /identity/v3/projects/cc99db8fa2bf4a61958c84f1331aa81a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:33.525675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26759876-0827-46d5-8383-eb3750a17b48 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.526104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26759876-0827-46d5-8383-eb3750a17b48 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.526505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26759876-0827-46d5-8383-eb3750a17b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.529385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] MFA Rules not processed for user `90f722382aea480da062c1bec48d6b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:33.530600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.546646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-26759876-0827-46d5-8383-eb3750a17b48 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:33.547981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-26759876-0827-46d5-8383-eb3750a17b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.596588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1018/2038] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:33.608922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.610481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.614803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1021/2039] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:33.629290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.629582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.629857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.654924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:33.655949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.664471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1022/2040] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:33.668679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.673516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b459ab-fbbb-44cc-8608-23718208f7f4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.677806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.678445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.679309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.679309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.683534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b17941-7c9c-49af-9b62-f1e0af7acd6f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.689643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.689737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.690070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.690242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.690335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.690523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.691167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:33.691504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.693085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.705802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1023/2041] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:33:33.707376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1019/2042] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:33.712565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.724118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f60a87e-3c58-473d-b4e0-799e6d22eb4f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.728450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.730208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:33.732255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.732874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.733157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.733769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.734605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.739648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.746548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a791a7-6084-40d4-bdcf-2b283b80cd9c service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.752315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.752860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.753996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.753996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] PATH_INFO: `/v3/users/90f722382aea480da062c1bec48d6b6c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.754677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90f722382aea480da062c1bec48d6b6c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.756692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.869366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1024/2043] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:33 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:33.873283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:33.933734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d06ec-25a8-4b32-a322-eff2327b45e9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:33.938666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:33.940206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:33.940566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:33.940772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:33.941412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:33.943413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:33.960372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90f722382aea480da062c1bec48d6b6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.972599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90f722382aea480da062c1bec48d6b6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.979313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90f722382aea480da062c1bec48d6b6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:33.980693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90f722382aea480da062c1bec48d6b6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:34.055695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a733c444-f41c-4b92-bfe2-f59781ac6ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90f722382aea480da062c1bec48d6b6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:34.064691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1020/2044] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:33 2022] DELETE /identity/v3/users/90f722382aea480da062c1bec48d6b6c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:34.067991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1025/2045] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:33 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:34.069263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.070041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.070470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.070885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.071823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:34.094268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb460a-bb5d-4176-bcdf-2cf594942667 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.094412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:34.095548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:34.149492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.150104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.150440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.150716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.151394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.152990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.158162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1021/2046] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:34.163185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1026/2047] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:34.166808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.168070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.169706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:34.177280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e545818-b78b-4ce8-a9e4-689b4eab401b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.182010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.182612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.183023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.183357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.184058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.185687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.196978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1022/2048] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:34.201086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.214373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a0842-9ee0-481c-ad5a-97b0b28abd6e admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.215930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.216575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.216878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.217121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] PATH_INFO: `/v3/projects/950639f30ef54fd0ba235d82a5e5eac2/users/7a81dfb02c15460e8913021782ae5f56/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.228006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace07b3-e417-42af-ad3e-ea05eb39d65f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.231909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1edcb-4a75-4947-87e2-4d7ea0e9821c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.237376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950639f30ef54fd0ba235d82a5e5eac2, user_id=7a81dfb02c15460e8913021782ae5f56, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.237496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1edcb-4a75-4947-87e2-4d7ea0e9821c service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.237496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1edcb-4a75-4947-87e2-4d7ea0e9821c service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.237496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1edcb-4a75-4947-87e2-4d7ea0e9821c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.237757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.246234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1edcb-4a75-4947-87e2-4d7ea0e9821c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.247915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1edcb-4a75-4947-87e2-4d7ea0e9821c service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.258136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1027/2049] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:34.307973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1023/2050] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:34 2022] PUT /identity/v3/projects/950639f30ef54fd0ba235d82a5e5eac2/users/7a81dfb02c15460e8913021782ae5f56/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:34.319651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.319936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.320208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.339605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:34.350256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] MFA Rules not processed for user `7a81dfb02c15460e8913021782ae5f56`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:34.351641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:34.415805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1028/2051] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:34.420317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.421341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:34.500439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dbf35-79ed-4225-b67a-a07c7c8cd103 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.511693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.511693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.511693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.511693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.512006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.515606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.594655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:34.632333 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1024/2052] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:34 2022] POST /identity/v3/projects => generated 379 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:34.633374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.646265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6bdc8-957e-4dcf-8f44-d2f87af95d51 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.715086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.716944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:34.722741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.723557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.723840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.724110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.724736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.726668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.733239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac7b3f-b6e7-408a-8d97-575e51d4b3de admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.738438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.739066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.739369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.739667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] PATH_INFO: `/v3/projects/d83e4a2162f04e55a75f21f9b65bce5a` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.741601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83e4a2162f04e55a75f21f9b65bce5a)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.744007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.808705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1029/2053] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:34 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:34.810568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.821563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429beec5-ac81-4343-8346-a10bfe92c1de admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.826173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.826763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.827048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.827354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.828126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.829707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.843509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1030/2054] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:34.844498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.854608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038e4e8-07b2-478b-85de-750d33459550 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.865860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83e4a2162f04e55a75f21f9b65bce5a', 'request_id': 'req-50f1191e-d98a-4015-92b3-c18fde6f5040'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:34.872675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83e4a2162f04e55a75f21f9b65bce5a', 'request_id': 'req-50f1191e-d98a-4015-92b3-c18fde6f5040'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:34.879781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83e4a2162f04e55a75f21f9b65bce5a', 'request_id': 'req-50f1191e-d98a-4015-92b3-c18fde6f5040'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:34.915735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.916739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.917085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.917427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.918153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.920019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:34.931307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1031/2055] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:34.935535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:34.949707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d4264-94d2-46de-9b10-14bbca8ec11b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:34.954488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d24811-5238-45a6-8b47-3059f2cf46a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:34.955890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d24811-5238-45a6-8b47-3059f2cf46a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:34.956176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d24811-5238-45a6-8b47-3059f2cf46a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:34.956534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d24811-5238-45a6-8b47-3059f2cf46a1 admin admin] PATH_INFO: `/v3/projects/ef9a3a8113504cf39a6eeca4073600cb/users/2585d179e6de4798bfcc51bb8c3f20dc/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:34.972006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d24811-5238-45a6-8b47-3059f2cf46a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef9a3a8113504cf39a6eeca4073600cb, user_id=2585d179e6de4798bfcc51bb8c3f20dc, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:34.982423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d24811-5238-45a6-8b47-3059f2cf46a1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.007769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] The token cache is being invalidate because project d83e4a2162f04e55a75f21f9b65bce5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:35.008063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:35.017128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-50f1191e-d98a-4015-92b3-c18fde6f5040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:35.017128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1025/2056] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:34 2022] DELETE /identity/v3/projects/d83e4a2162f04e55a75f21f9b65bce5a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:35.017128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa735-2e01-4f40-999b-b5b8031e1226 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.017128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa735-2e01-4f40-999b-b5b8031e1226 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.017128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fa735-2e01-4f40-999b-b5b8031e1226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.036698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-6b2fa735-2e01-4f40-999b-b5b8031e1226 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:35.038203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6b2fa735-2e01-4f40-999b-b5b8031e1226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:35.046801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1026/2057] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:35.056576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.058970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.059281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.074096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1032/2058] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:34 2022] PUT /identity/v3/projects/ef9a3a8113504cf39a6eeca4073600cb/users/2585d179e6de4798bfcc51bb8c3f20dc/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:35.081588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:35.082870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:35.085322 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.085613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.085888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.112595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] MFA Rules not processed for user `2585d179e6de4798bfcc51bb8c3f20dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:35.113806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:35.117405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:35.143607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1027/2059] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:35.147806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.159201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e28e4d-48af-4d40-9b8e-153e76b4ebe5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.164686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.165432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.165776 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.166115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.166888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:35.167267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.168986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.180135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:35.183761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1028/2060] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:35.188030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.199397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739697-4aa3-4f93-80da-994158b189fa admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.209998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.209998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.209998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.209998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.210304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.215647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1033/2061] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:35.215647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.215901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.217071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:35.230600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e673b33-54be-4638-8672-4c043faa0fce None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.235928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.241058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.241058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.241058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.241058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.241058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.364434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1029/2062] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:35.365328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.389665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1034/2063] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:35.396840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.436828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6432e90-7374-4452-b163-b082848e27c8 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.479829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac524b-8e72-4183-91a8-953f13d00378 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.485132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.486615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.486893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.487158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.487823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.490445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.502314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.503120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.503437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.503707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.504335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.506250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.576661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1035/2064] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:35.580833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.591087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e18f8c-41fa-4e2c-9993-27c47e5e56c2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.596324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.596902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.597177 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.597442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.598064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.599664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.611000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1036/2065] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:35.616178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.620702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1030/2066] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/users => generated 283 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:35.624314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.634951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712ac91-ac31-47e5-9e87-8a76f97c7602 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.635171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4f409-c5ab-4879-a807-b8f76c6e75c0 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.640589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.641309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.641626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.641930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] PATH_INFO: `/v3/projects/2a71241a6c04420f80ab251914d628c7/users/d7ad95b7467f45adb0b1526d47ae273d/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.646105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.653135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.653417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.653692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.654365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.656019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.659926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a71241a6c04420f80ab251914d628c7, user_id=d7ad95b7467f45adb0b1526d47ae273d, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.662692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.670099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1031/2067] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:35.677320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.681596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb3a69-337f-43b1-bee9-ac5569c25da8 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.686540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.687133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.687461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.687732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.688381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.690656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.701341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1032/2068] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:35.706271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.716404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6adc9d-96ec-4880-916c-36a873492003 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.722491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566ce3e-3fea-4803-af11-0767233f1e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.723141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566ce3e-3fea-4803-af11-0767233f1e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.723695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566ce3e-3fea-4803-af11-0767233f1e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.729080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566ce3e-3fea-4803-af11-0767233f1e51 admin admin] PATH_INFO: `/v3/projects/aa32873224e04d338c3acb912289781f/users/e97ab3e1e9364cf990d1eb17ee486d06/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.738497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566ce3e-3fea-4803-af11-0767233f1e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa32873224e04d338c3acb912289781f, user_id=e97ab3e1e9364cf990d1eb17ee486d06, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.741222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566ce3e-3fea-4803-af11-0767233f1e51 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.754058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1037/2069] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:35 2022] PUT /identity/v3/projects/2a71241a6c04420f80ab251914d628c7/users/d7ad95b7467f45adb0b1526d47ae273d/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:35.758173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.820032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9176a-f2a1-4e37-a61c-e6cdc52bac20 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.825000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.825595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.825869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.826139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.826793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.828442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.839162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1038/2070] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:35.843250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:35.852964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab65a6-4cd3-4467-b7c6-aaf5ebbb1d22 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:35.857691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee003509-429c-4253-8ce9-097669f64dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:35.858311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee003509-429c-4253-8ce9-097669f64dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.858590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee003509-429c-4253-8ce9-097669f64dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.858860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee003509-429c-4253-8ce9-097669f64dd1 admin admin] PATH_INFO: `/v3/projects/2a71241a6c04420f80ab251914d628c7/users/d7ad95b7467f45adb0b1526d47ae273d/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.860780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee003509-429c-4253-8ce9-097669f64dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a71241a6c04420f80ab251914d628c7, user_id=d7ad95b7467f45adb0b1526d47ae273d, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:35.863497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee003509-429c-4253-8ce9-097669f64dd1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:35.874042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1039/2071] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:35 2022] PUT /identity/v3/projects/2a71241a6c04420f80ab251914d628c7/users/d7ad95b7467f45adb0b1526d47ae273d/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:35.878097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1033/2072] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:35 2022] PUT /identity/v3/projects/aa32873224e04d338c3acb912289781f/users/e97ab3e1e9364cf990d1eb17ee486d06/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:35.888671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56868-9920-46c9-a851-399ab718b74c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.888827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56868-9920-46c9-a851-399ab718b74c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.889092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a56868-9920-46c9-a851-399ab718b74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.890761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:35.891220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:35.891681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:35.922759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] MFA Rules not processed for user `d7ad95b7467f45adb0b1526d47ae273d`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:35.923058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-81a56868-9920-46c9-a851-399ab718b74c None None] MFA Rules not processed for user `e97ab3e1e9364cf990d1eb17ee486d06`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:35.926392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-81a56868-9920-46c9-a851-399ab718b74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:35.926532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:36.003050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1040/2073] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:36.004650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:36.015308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1034/2074] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:36.015437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a56868-9920-46c9-a851-399ab718b74c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:36.016714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-81a56868-9920-46c9-a851-399ab718b74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:36.023426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:36.024864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:36.077046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a56868-9920-46c9-a851-399ab718b74c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:36.082138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:36.083049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:36.083576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:36.084054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:36.094942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:36.096962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:36.104637 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18e9b9-a358-40f7-9866-f4afb42da18b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:36.107266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1041/2075] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:36.109769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:36.110691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:36.111495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:36.112284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:36.125584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:36.127983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:36.138625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1035/2076] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:36.425164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:36.505803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:36.514195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17facf9-1c4c-430d-937e-f64b2cfe959b service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:36.517858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:36.518432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:36.518706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:36.518972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:36.520692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:36.578739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:36.580275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:36.589066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1042/2077] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:36.718133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:37.644439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:37.785111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:40.778634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:40.796632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dd3b9-3c59-449a-9817-a6becf2de024 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:40.801256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:40.801984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:40.802457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:40.802743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:40.804887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:40.886752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:40.888373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfbc3bc-7db0-458c-8458-6a529c261eac service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:40.899973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1036/2078] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:48.304840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=43 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:49.142773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:49.144287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:49.217135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d94a26-437e-4245-afd1-82bd39a12d38 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:49.222504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:49.223133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:49.223445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:49.223722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] PATH_INFO: `/v3/users/648b19a74b394e7183b399fea981907e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:49.224922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648b19a74b394e7183b399fea981907e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:49.226909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:49.446400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648b19a74b394e7183b399fea981907e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:49.460592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648b19a74b394e7183b399fea981907e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:49.461310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648b19a74b394e7183b399fea981907e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:49.473319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648b19a74b394e7183b399fea981907e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:49.572579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648b19a74b394e7183b399fea981907e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:49.579641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1043/2079] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:49 2022] DELETE /identity/v3/users/648b19a74b394e7183b399fea981907e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:49.585347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:49.585831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:49.586079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:49.606294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:49.610992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:49.661915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:49.670774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1037/2080] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:49.681410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:49.682898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:49.744564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fef200-845f-418f-8dbc-51b641ff37c0 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:49.748380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:49.749054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:49.749402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:49.749742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:49.759021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:49.760547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:49.770648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1044/2081] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:49.980169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:49.981615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:49.994519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78937f8-76cb-42e2-abac-c7459c506182 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:49.999421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:50.000223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.000543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.000916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] PATH_INFO: `/v3/users/7a81dfb02c15460e8913021782ae5f56` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.002896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a81dfb02c15460e8913021782ae5f56)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:50.005089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:50.190797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a81dfb02c15460e8913021782ae5f56'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.211671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a81dfb02c15460e8913021782ae5f56'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.212969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:50.214925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a81dfb02c15460e8913021782ae5f56'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.215167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:50.228465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f51998-3495-4ab0-9a46-f26ebddf5594 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:50.231409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a81dfb02c15460e8913021782ae5f56'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.234024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:50.234651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.235057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.235380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] PATH_INFO: `/v3/projects/ab5c861a64844b068e8798410fa5054a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.236671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5c861a64844b068e8798410fa5054a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:50.239071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:50.260903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:50.302379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-7243cc37-fd34-472a-bdc2-8b5153b4abe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a81dfb02c15460e8913021782ae5f56'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.308804 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1038/2082] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:49 2022] DELETE /identity/v3/users/7a81dfb02c15460e8913021782ae5f56 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:50.313130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.313437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.313768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.334439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:50.336107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:50.379881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5c861a64844b068e8798410fa5054a', 'request_id': 'req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.386534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5c861a64844b068e8798410fa5054a', 'request_id': 'req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.393003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5c861a64844b068e8798410fa5054a', 'request_id': 'req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.401043 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1039/2083] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:50.414048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:50.414048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:50.476708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] The token cache is being invalidate because project ab5c861a64844b068e8798410fa5054a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:50.477247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.477811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-27bcc203-3195-41b0-9bdf-0d84d0fc0cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:50.479812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1045/2084] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:50 2022] DELETE /identity/v3/projects/ab5c861a64844b068e8798410fa5054a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:50.484315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8b09-ee64-4d1b-aed7-dae35e59f38a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.484601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8b09-ee64-4d1b-aed7-dae35e59f38a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.485003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb8b09-ee64-4d1b-aed7-dae35e59f38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.493079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced261b9-867b-4a47-af61-4ff68419073e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:50.512627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-2cfb8b09-ee64-4d1b-aed7-dae35e59f38a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:50.514356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2cfb8b09-ee64-4d1b-aed7-dae35e59f38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:50.523031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:50.551488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:50.552295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.552677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.553058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.556212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:50.582229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1046/2085] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:50.590303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.590690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.591025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.614571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:50.614571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:50.625790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1047/2086] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:50.627695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:50.629387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:50.630622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:50.639480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1040/2087] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:50.639716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67246a60-9dba-45e1-8029-24b22f24c095 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:50.653870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:50.654931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.655330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.655794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.656549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:50.656968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:50.659201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:50.673264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1048/2088] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:33:50.685913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:50.691447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508aa7-fe4e-47fe-b71b-3fbb65db237c service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:50.696805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:50.697425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.697719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.697970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.698816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:50.700774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:50.815669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1041/2089] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:50 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:50.819958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:50.834526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe854f1-6bc6-4d24-82ff-d1885bc4097a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:50.914375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:50.914993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:50.916969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:50.916969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:50.916969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:50.918360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.083690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.090782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1049/2090] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:50 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:51.106341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331530f-9398-4c9e-b5ab-71a82ff474c6 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.112095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.112712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.112991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.113267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.114067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.115693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.129508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1042/2091] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:51.130367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.141605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a8212-40b7-4c36-a5ce-24d49506b749 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.152628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.153409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.153774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.154922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.175062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.180155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.204028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1050/2092] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:51.208431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.221297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aa0a9-e697-4212-b0ae-b780e4911a22 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.230052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f92092-c136-4cd2-8fe9-13c6b279b818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.230728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f92092-c136-4cd2-8fe9-13c6b279b818 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.231013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f92092-c136-4cd2-8fe9-13c6b279b818 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.231364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f92092-c136-4cd2-8fe9-13c6b279b818 admin admin] PATH_INFO: `/v3/projects/1ba0b09aa1bc4f898ae9e97f6b87f5f7/users/841bc9717d6247888cda5118d5e490ed/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.256030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.260886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:51.276511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f92092-c136-4cd2-8fe9-13c6b279b818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba0b09aa1bc4f898ae9e97f6b87f5f7, user_id=841bc9717d6247888cda5118d5e490ed, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.279437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f92092-c136-4cd2-8fe9-13c6b279b818 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.285998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:51.290628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c88eb-db1d-415f-848d-eae2b0b6f2bd admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.305076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.306070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.306518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.306947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] PATH_INFO: `/v3/projects/950639f30ef54fd0ba235d82a5e5eac2` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.308694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] RBAC: Authorizing `identity:delete_project(project_id=950639f30ef54fd0ba235d82a5e5eac2)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.311909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.398411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1043/2093] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:51 2022] PUT /identity/v3/projects/1ba0b09aa1bc4f898ae9e97f6b87f5f7/users/841bc9717d6247888cda5118d5e490ed/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:51.411081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.412444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.413360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.437839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950639f30ef54fd0ba235d82a5e5eac2', 'request_id': 'req-108249d3-8bd1-41d6-a00e-30261f1db19f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.443490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] MFA Rules not processed for user `841bc9717d6247888cda5118d5e490ed`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:51.444253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950639f30ef54fd0ba235d82a5e5eac2', 'request_id': 'req-108249d3-8bd1-41d6-a00e-30261f1db19f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.444720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:51.450458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950639f30ef54fd0ba235d82a5e5eac2', 'request_id': 'req-108249d3-8bd1-41d6-a00e-30261f1db19f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.509039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1044/2094] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:51.509886 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.511165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:51.523874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:51.532180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] The token cache is being invalidate because project 950639f30ef54fd0ba235d82a5e5eac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:51.532720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.533671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.536432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1051/2095] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:51 2022] DELETE /identity/v3/projects/950639f30ef54fd0ba235d82a5e5eac2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:51.541188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.541188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:51.576984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd43a8-0a47-4190-9af4-cbf2035b593d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.581667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.582295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.582570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.582838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] PATH_INFO: `/v3/users/26b3d455a51d4e1d85aac4903d5f72d0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.583985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b3d455a51d4e1d85aac4903d5f72d0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.586284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.602261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108249d3-8bd1-41d6-a00e-30261f1db19f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.607440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.608046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.608326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.608600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.609294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.611250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.731846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1052/2096] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:51 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:51.732743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.734290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:51.797109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71eb7-4dd3-4a13-8bde-727ae9e93aa1 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:51.801951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:51.802755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:51.803029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:51.803329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] PATH_INFO: `/v3/users/2585d179e6de4798bfcc51bb8c3f20dc` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:51.804445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2585d179e6de4798bfcc51bb8c3f20dc)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:51.806412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:51.835608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b3d455a51d4e1d85aac4903d5f72d0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.847764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b3d455a51d4e1d85aac4903d5f72d0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.848441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b3d455a51d4e1d85aac4903d5f72d0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.855503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b3d455a51d4e1d85aac4903d5f72d0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.859719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:51.957288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b3d455a51d4e1d85aac4903d5f72d0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:51.961592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1045/2097] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:51 2022] DELETE /identity/v3/users/26b3d455a51d4e1d85aac4903d5f72d0 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:51.962480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:51.965529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.031700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a0b58-d0d4-4097-bef4-68791fe4a6d7 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.037382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37648b27-1b22-4b01-b473-eef746f5a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.038021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37648b27-1b22-4b01-b473-eef746f5a3dc admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.038303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37648b27-1b22-4b01-b473-eef746f5a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.038578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37648b27-1b22-4b01-b473-eef746f5a3dc admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.039207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37648b27-1b22-4b01-b473-eef746f5a3dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.041184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37648b27-1b22-4b01-b473-eef746f5a3dc admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.044717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2585d179e6de4798bfcc51bb8c3f20dc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.057279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2585d179e6de4798bfcc51bb8c3f20dc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.057889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2585d179e6de4798bfcc51bb8c3f20dc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.064598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2585d179e6de4798bfcc51bb8c3f20dc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.109479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1046/2098] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:51 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:52.110711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.111108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.111429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.126593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2585d179e6de4798bfcc51bb8c3f20dc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.130487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:52.135602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.135738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1053/2099] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:51 2022] DELETE /identity/v3/users/2585d179e6de4798bfcc51bb8c3f20dc => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:52.135738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.136258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.146055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:52.198337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1047/2100] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:52.210914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.212672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.214049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739dfc-d8a6-4a08-a54d-a0e2db52a91e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.219500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.220233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.220756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.221082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.221786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.223693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.241125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1054/2101] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:52.246040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.257275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262f171-e200-4271-80b1-db607142a7ec admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.262034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.262699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.262983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.263274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.263913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.265543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.276344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1055/2102] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:52.277632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c40db8-3c4c-4d24-96db-a3d01b2c56c8 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.280557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.283046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f6cd8-31d2-41a0-a9bd-b2dd65a4352a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.283046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f6cd8-31d2-41a0-a9bd-b2dd65a4352a service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.283046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f6cd8-31d2-41a0-a9bd-b2dd65a4352a service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.283046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f6cd8-31d2-41a0-a9bd-b2dd65a4352a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.293118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c65fc-2d5c-4618-92a4-ec8d88e0b20d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.296235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f6cd8-31d2-41a0-a9bd-b2dd65a4352a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.297801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f6cd8-31d2-41a0-a9bd-b2dd65a4352a service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.297915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.298549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.298827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.299090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] PATH_INFO: `/v3/projects/5a4d850b78324fdda1d1027e525cf2f3/users/68ba323ca9454d28a815d18267021905/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.311789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1048/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:52.314982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4d850b78324fdda1d1027e525cf2f3, user_id=68ba323ca9454d28a815d18267021905, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.318372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.395399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1056/2104] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:52 2022] PUT /identity/v3/projects/5a4d850b78324fdda1d1027e525cf2f3/users/68ba323ca9454d28a815d18267021905/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:52.408921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.409395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.409799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.440510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] MFA Rules not processed for user `68ba323ca9454d28a815d18267021905`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:52.442648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.524113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1049/2105] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:52.528671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.595012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:52.605701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f8e0eb-6333-4e81-93ed-ca876a021be9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.610406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.611007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.611367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.611649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.612288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.614860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.676150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.680720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.694210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c0eba-f430-4460-8422-cbf274e94b49 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.699158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.699901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.700155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.700389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] PATH_INFO: `/v3/projects/ef9a3a8113504cf39a6eeca4073600cb` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.701525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef9a3a8113504cf39a6eeca4073600cb)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.703497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.740972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1057/2106] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:52 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:52.742283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.807718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b76b4-34e1-4044-a615-c2d59c898692 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:52.812712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:52.813382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:52.813669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:52.814026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:52.814560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:52.817562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:52.842266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef9a3a8113504cf39a6eeca4073600cb', 'request_id': 'req-eaab1553-8aaa-43a5-96cc-a466fdcf5592'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.847740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef9a3a8113504cf39a6eeca4073600cb', 'request_id': 'req-eaab1553-8aaa-43a5-96cc-a466fdcf5592'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.853744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef9a3a8113504cf39a6eeca4073600cb', 'request_id': 'req-eaab1553-8aaa-43a5-96cc-a466fdcf5592'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.933143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] The token cache is being invalidate because project ef9a3a8113504cf39a6eeca4073600cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:52.933431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.934207 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1058/2107] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:52 2022] POST /identity/v3/users => generated 298 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:52.934327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:52.934923 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.936055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1050/2108] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:52 2022] DELETE /identity/v3/projects/ef9a3a8113504cf39a6eeca4073600cb => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:52.936523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.939911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:52.941347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:52.948979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c225a-039c-4457-8af5-02aa610063e2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.006828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.007497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.007780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.008058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] PATH_INFO: `/v3/projects/1d370ba54d034a819ac152cadfbdb636` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.008720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab1553-8aaa-43a5-96cc-a466fdcf5592 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.009361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d370ba54d034a819ac152cadfbdb636)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.011317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.013955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.014598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.014972 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.015334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.016042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.017681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.031467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1051/2109] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:53.031467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:53.038904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:53.049238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:53.054309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d17f-2ecc-4f16-8ef4-efd355055ad6 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.058113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.058728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.059000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.059297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] PATH_INFO: `/v3/users/d7ad95b7467f45adb0b1526d47ae273d` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.060421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7ad95b7467f45adb0b1526d47ae273d)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.062318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.119677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d370ba54d034a819ac152cadfbdb636', 'request_id': 'req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.125224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d370ba54d034a819ac152cadfbdb636', 'request_id': 'req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.131373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d370ba54d034a819ac152cadfbdb636', 'request_id': 'req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.262541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] The token cache is being invalidate because project 1d370ba54d034a819ac152cadfbdb636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:53.262541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.262541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.262541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1059/2110] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:52 2022] DELETE /identity/v3/projects/1d370ba54d034a819ac152cadfbdb636 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:53.262541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:53.262957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:53.293899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7ad95b7467f45adb0b1526d47ae273d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.312110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7ad95b7467f45adb0b1526d47ae273d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.312955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7ad95b7467f45adb0b1526d47ae273d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.320943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7ad95b7467f45adb0b1526d47ae273d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.333209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3ef98-f3c8-4666-8433-c6029a7bde1e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.338760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.339445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.339727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.347886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] PATH_INFO: `/v3/projects/ebb6e86c1a3e413ba95bc0b339b5f6e5/users/a1e817cb0b154cb7b740f8a92fcbc144/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.355185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb6e86c1a3e413ba95bc0b339b5f6e5, user_id=a1e817cb0b154cb7b740f8a92fcbc144, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.357868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.430202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7ad95b7467f45adb0b1526d47ae273d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.445111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1052/2111] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:53 2022] DELETE /identity/v3/users/d7ad95b7467f45adb0b1526d47ae273d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:53.449261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:53.451387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:53.466894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1060/2112] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:53 2022] PUT /identity/v3/projects/ebb6e86c1a3e413ba95bc0b339b5f6e5/users/a1e817cb0b154cb7b740f8a92fcbc144/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:53.475489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:53.478680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:53.558560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fb66f-16a8-4b02-8942-e538c350a310 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.636104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2b399-d7cc-4a3c-a6ff-dc5ca6cc876e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.641340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.642537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.642818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.643095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.643947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.654123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.668138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1061/2113] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:53.672251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:53.687981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaaf25c-d29c-4c5d-aa91-f2ea585c62e5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.689686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.690421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.690702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.691065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] PATH_INFO: `/v3/users/4c469e758c8d48e5950ca7632be0287b` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.694874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c469e758c8d48e5950ca7632be0287b)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.696897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.702527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c565f-569c-40ba-9352-18541d981dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.704316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c565f-569c-40ba-9352-18541d981dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.704316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c565f-569c-40ba-9352-18541d981dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.704316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c565f-569c-40ba-9352-18541d981dbe admin admin] PATH_INFO: `/v3/projects/ebb6e86c1a3e413ba95bc0b339b5f6e5/users/a1e817cb0b154cb7b740f8a92fcbc144/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.711396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c565f-569c-40ba-9352-18541d981dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb6e86c1a3e413ba95bc0b339b5f6e5, user_id=a1e817cb0b154cb7b740f8a92fcbc144, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:53.713891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c565f-569c-40ba-9352-18541d981dbe admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:53.747750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1062/2114] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:53 2022] PUT /identity/v3/projects/ebb6e86c1a3e413ba95bc0b339b5f6e5/users/a1e817cb0b154cb7b740f8a92fcbc144/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:53.752743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.753029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.753300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.787717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] MFA Rules not processed for user `a1e817cb0b154cb7b740f8a92fcbc144`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:53.792752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:53.881158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1063/2115] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:53.893307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:53.896600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:53.919674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c469e758c8d48e5950ca7632be0287b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.934899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c469e758c8d48e5950ca7632be0287b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.936129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c469e758c8d48e5950ca7632be0287b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.946913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c469e758c8d48e5950ca7632be0287b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:53.981795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb814e8-652d-4848-9568-4d48baa06e5a None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:53.985595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:53.986274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:53.986643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:53.986985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:53.988913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:54.022989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c469e758c8d48e5950ca7632be0287b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:54.030433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1053/2116] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:53 2022] DELETE /identity/v3/users/4c469e758c8d48e5950ca7632be0287b => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:54.064842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:54.067075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:54.078264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1064/2117] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:33:54.252786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:54.317415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:54.366839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:54.368812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:54.455803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9a86d-61e3-4074-8bbf-828854af9587 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:54.471391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:54.473599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:54.474855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:54.476545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] PATH_INFO: `/v3/projects/2a71241a6c04420f80ab251914d628c7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:54.477933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a71241a6c04420f80ab251914d628c7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:54.480020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:54.608056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a71241a6c04420f80ab251914d628c7', 'request_id': 'req-f936d637-ec62-47e4-9a7e-c7417449998b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:54.614373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a71241a6c04420f80ab251914d628c7', 'request_id': 'req-f936d637-ec62-47e4-9a7e-c7417449998b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:54.620715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a71241a6c04420f80ab251914d628c7', 'request_id': 'req-f936d637-ec62-47e4-9a7e-c7417449998b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:54.702074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] The token cache is being invalidate because project 2a71241a6c04420f80ab251914d628c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:54.703117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:54.703607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:54.704119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f936d637-ec62-47e4-9a7e-c7417449998b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:54.711192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1054/2118] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:54 2022] DELETE /identity/v3/projects/2a71241a6c04420f80ab251914d628c7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:54.712892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fc7fb-9319-4d49-b314-aac371e77a64 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:54.713290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fc7fb-9319-4d49-b314-aac371e77a64 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:54.713662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fc7fb-9319-4d49-b314-aac371e77a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:54.752799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a21fc7fb-9319-4d49-b314-aac371e77a64 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:54.757363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a21fc7fb-9319-4d49-b314-aac371e77a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:54.780232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d143f-b3d6-4664-83c8-7baecd9f6e70 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:54.882205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:54.882963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:54.883450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:54.883809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] PATH_INFO: `/v3/projects/1227885f866b4b98b38ebe72008428b1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:54.885790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1227885f866b4b98b38ebe72008428b1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:54.887296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1055/2119] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:54.888745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:54.897818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:54.898784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:54.899220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:54.972299 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:54.974453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:54.990861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1056/2120] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:54.993502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.004302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a0303-fe4a-4152-b8e8-9e49efce3c1a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.009830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.010518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.010825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.011414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.012245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:55.012546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.014307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.028157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1057/2121] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:33:55.032159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.038490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1227885f866b4b98b38ebe72008428b1', 'request_id': 'req-87ff9d88-b291-4e02-8468-52ce4e77c7b5'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:55.043960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1a3d5-c097-48dc-b282-7a99d374ef4a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.044176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1227885f866b4b98b38ebe72008428b1', 'request_id': 'req-87ff9d88-b291-4e02-8468-52ce4e77c7b5'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:55.050971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1227885f866b4b98b38ebe72008428b1', 'request_id': 'req-87ff9d88-b291-4e02-8468-52ce4e77c7b5'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:55.103744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.104351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.104630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.104902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.105525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.107478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.164741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] The token cache is being invalidate because project 1227885f866b4b98b38ebe72008428b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:55.169340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:55.169340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87ff9d88-b291-4e02-8468-52ce4e77c7b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:55.169340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1065/2122] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:54 2022] DELETE /identity/v3/projects/1227885f866b4b98b38ebe72008428b1 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:55.171888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279537f2-41f0-4bd3-9fc9-74734af95de7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.172172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279537f2-41f0-4bd3-9fc9-74734af95de7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.172446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279537f2-41f0-4bd3-9fc9-74734af95de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.193138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-279537f2-41f0-4bd3-9fc9-74734af95de7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:55.194407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-279537f2-41f0-4bd3-9fc9-74734af95de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:55.212099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1066/2123] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:55.216790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.217105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.217378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.237978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1058/2124] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/projects => generated 369 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:55.241199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:55.242344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:55.242475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.244345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:55.331269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e76bd0-0b56-43e0-9202-9c52aa0a585f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.336032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1067/2125] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:33:55.339985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.340614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.340894 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.341163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.341815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.342335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.343982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.354501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475651cc-23d9-496a-9a24-8373774ac98b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.375251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.375899 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.376184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.376629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.377452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:55.377762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.381198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.397122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1068/2126] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:55.401198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.418524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d66978-e4a5-4995-ae6f-3161dab1beb5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.425803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.426464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.426741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.427512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.430239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.430525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.458842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1059/2127] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/users => generated 285 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:55.467417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.474994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45cb9d-c6da-40f5-bb5a-5b9f53a612fb admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.480412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.481198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.481548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.481887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.482613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.484505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.495594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1060/2128] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:55.499462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.510292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639fa8b-a0c0-4c82-94be-4c0bdf2e3a93 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.514982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.520099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.520099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.520099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.520099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.520099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.529344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1061/2129] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:55.533209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.545084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461e6e5-8a1a-4b5a-a9f8-7f8a4052aa94 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.551104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cabd393-1489-4bb4-80c5-8fff4db879bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.552006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabd393-1489-4bb4-80c5-8fff4db879bd admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.552288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabd393-1489-4bb4-80c5-8fff4db879bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.552570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cabd393-1489-4bb4-80c5-8fff4db879bd admin admin] PATH_INFO: `/v3/projects/61e7409c7b6e4520971aabf4a6fd76e1/users/805a8f0541914c4eb06aa0dd20bf8cc6/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.554921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1069/2130] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:55.558774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.569453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabd393-1489-4bb4-80c5-8fff4db879bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e7409c7b6e4520971aabf4a6fd76e1, user_id=805a8f0541914c4eb06aa0dd20bf8cc6, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.572222 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cabd393-1489-4bb4-80c5-8fff4db879bd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.628986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003c26f-c9dc-4a16-9075-20bfdf1d28fc admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.649676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.650505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.650792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.651065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.651743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.654385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.665295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1062/2131] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:55 2022] PUT /identity/v3/projects/61e7409c7b6e4520971aabf4a6fd76e1/users/805a8f0541914c4eb06aa0dd20bf8cc6/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:55.677756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.678046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.678355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.715184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] MFA Rules not processed for user `805a8f0541914c4eb06aa0dd20bf8cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:55.716672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:55.773787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1070/2132] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/users => generated 282 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:55.774726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:55.778375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.799874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee31293-be3d-46b1-af18-046eb51dde74 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.830546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1063/2133] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:55.843739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.845679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:55.866195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:55.934721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.935393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.938140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.938140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.938140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.941687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.950233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eaa28-2ccd-4424-bf6d-4a2fd4843e6a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:55.956754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1071/2134] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:55.956981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:55.956981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:55.956981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:55.956981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:55.958566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:55.978540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:55.981910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:55.982663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d5834-73e2-4337-b6ff-bae92fec7f0a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.005672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d51f4-7d0c-4896-af2e-84d1b1b115af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.006598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d51f4-7d0c-4896-af2e-84d1b1b115af admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.006891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d51f4-7d0c-4896-af2e-84d1b1b115af admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.007170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d51f4-7d0c-4896-af2e-84d1b1b115af admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.009564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d51f4-7d0c-4896-af2e-84d1b1b115af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:56.012910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1064/2135] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:56.013642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d51f4-7d0c-4896-af2e-84d1b1b115af admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:56.027274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1072/2136] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:33:56.033206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:56.046732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d8f964-553f-48c0-819a-94c20460e5d3 service glance] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.053002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.053689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.054022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.054306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] PATH_INFO: `/v3/projects/82abb82e7ee24c03ac62c14690857553/users/b200e51e32204cd58dd8c9561f30867f/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.072594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] RBAC: Authorizing `identity:create_grant(project_id=82abb82e7ee24c03ac62c14690857553, user_id=b200e51e32204cd58dd8c9561f30867f, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:56.075536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:56.201151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1065/2137] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:56 2022] PUT /identity/v3/projects/82abb82e7ee24c03ac62c14690857553/users/b200e51e32204cd58dd8c9561f30867f/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:56.213003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.213299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.213577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.247646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] MFA Rules not processed for user `b200e51e32204cd58dd8c9561f30867f`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:56.249162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:56.341895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1073/2138] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:56.346714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:56.428934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548efec-3033-49f2-9119-5af8353c906a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.433853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.434529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.434874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.435217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.435945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:56.438218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:56.528065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1066/2139] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:56 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:33:56.531424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:56.543934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:56.545509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c00e7-499c-4d26-95b3-bf12fdb4ac79 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.602360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.602975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.603271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.603543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.604274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:56.606058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:56.701033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1074/2140] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:56 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:56.701174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:56.708608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df9216-7cd0-4b4d-af9f-c923b9de1a34 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.713747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.714347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.714631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.714907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.715794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:56.721009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:56.733897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1067/2141] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:56.738654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:56.749040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54291120-b73f-40d0-b5af-935ef1e95884 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.753754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.754413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.754689 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:56.754961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] PATH_INFO: `/v3/projects/84114c87f813436db493d36ab2539c30/users/9d89903c76ef421fbda8774b63ffa621/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:56.769911 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84114c87f813436db493d36ab2539c30, user_id=9d89903c76ef421fbda8774b63ffa621, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:56.772610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:56.876805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1075/2142] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:56 2022] PUT /identity/v3/projects/84114c87f813436db493d36ab2539c30/users/9d89903c76ef421fbda8774b63ffa621/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:56.880651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:56.884136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:56.913272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710ba9-d7f7-4e71-8901-99572aff3392 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:56.986971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:56.998855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910222a6-4718-4beb-bda2-81d7850a6e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:56.998855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910222a6-4718-4beb-bda2-81d7850a6e16 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:56.998855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910222a6-4718-4beb-bda2-81d7850a6e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:57.000312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910222a6-4718-4beb-bda2-81d7850a6e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:57.000312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910222a6-4718-4beb-bda2-81d7850a6e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:57.001857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910222a6-4718-4beb-bda2-81d7850a6e16 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:57.013570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1068/2143] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:33:57.017460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:57.037266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b9d17-b1d0-4b22-a777-7173ae58e828 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:57.051502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:57.051502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:57.051502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:57.051502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] PATH_INFO: `/v3/projects/84114c87f813436db493d36ab2539c30/users/9d89903c76ef421fbda8774b63ffa621/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:57.051846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84114c87f813436db493d36ab2539c30, user_id=9d89903c76ef421fbda8774b63ffa621, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:57.055265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:57.073607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1076/2144] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:33:57 2022] PUT /identity/v3/projects/84114c87f813436db493d36ab2539c30/users/9d89903c76ef421fbda8774b63ffa621/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:57.101038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:57.101038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:57.101038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:57.141054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] MFA Rules not processed for user `9d89903c76ef421fbda8774b63ffa621`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:57.142372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:57.212760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1069/2145] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:57.226746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:57.229850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:57.293713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48d993-bfcd-4d17-b44e-766f672bc473 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:57.298466 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:57.299117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:57.299427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:57.299897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:57.312855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:57.314315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:57.326121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1077/2146] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:57.435851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:57.515276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:57.525371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:57.531376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632397a1-6d82-4922-8238-a55bb2b4e160 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:57.540318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:57.540948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:57.541227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:57.541506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:57.640093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:57.641569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:57.665453 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1070/2147] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:33:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:33:57.821503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:58.553069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:58.561817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:58.628957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b910b1-dd3d-423d-8704-32a5992f9cd1 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:58.633572 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:58.634210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:58.634495 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:58.634883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] PATH_INFO: `/v3/users/be18a802a38143518bb0bb79afaae011` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:58.636150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be18a802a38143518bb0bb79afaae011)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:58.638406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:58.965176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:58.969931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be18a802a38143518bb0bb79afaae011'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:58.981388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be18a802a38143518bb0bb79afaae011'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:58.982056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be18a802a38143518bb0bb79afaae011'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:58.988661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be18a802a38143518bb0bb79afaae011'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.069391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-02268df4-1de9-4053-8c1d-76527792a386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be18a802a38143518bb0bb79afaae011'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.075767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1078/2148] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:33:58 2022] DELETE /identity/v3/users/be18a802a38143518bb0bb79afaae011 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:33:59.333987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:59.346204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58092bf0-b09f-42fe-b0e5-11570a0186d7 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:59.433246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:59.433953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:59.434295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:59.434642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] PATH_INFO: `/v3/projects/7214cb5b7d4c4c949c84dc81601a89e0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:59.435879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7214cb5b7d4c4c949c84dc81601a89e0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:59.437857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:59.546780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:33:59.557600 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7214cb5b7d4c4c949c84dc81601a89e0', 'request_id': 'req-bfdc5c88-4084-4728-b6b3-31ca4948558b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.562977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7214cb5b7d4c4c949c84dc81601a89e0', 'request_id': 'req-bfdc5c88-4084-4728-b6b3-31ca4948558b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.569448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7214cb5b7d4c4c949c84dc81601a89e0', 'request_id': 'req-bfdc5c88-4084-4728-b6b3-31ca4948558b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.662303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] The token cache is being invalidate because project 7214cb5b7d4c4c949c84dc81601a89e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:33:59.662303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.662303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bfdc5c88-4084-4728-b6b3-31ca4948558b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:33:59.662303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1071/2149] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:33:59 2022] DELETE /identity/v3/projects/7214cb5b7d4c4c949c84dc81601a89e0 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:33:59.662773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:59.662773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:59.662773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:59.683275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:59.684497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:59.750797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1079/2150] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:59.759538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:59.759768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:59.760042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:59.779707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:33:59.780947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:33:59.789690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1072/2151] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:33:59.794077 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:59.804991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d90ff-a5bd-45c2-9346-514d8a0cc38f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:59.811190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:59.811462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:59.811721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:59.812060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:59.812841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:33:59.813143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:59.814729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:59.827298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1080/2152] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:33:59.832416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:33:59.841970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17e29e-c568-4598-8cca-954a66135f52 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:33:59.846662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:33:59.847371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:33:59.847741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:33:59.848012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:33:59.848668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:33:59.850563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:33:59.991128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1073/2153] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:33:59 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:33:59.995244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:00.005861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a105117-43c4-4585-bc7c-ae7e7953680b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:00.071374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:00.071979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:00.072261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:00.072532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:00.073163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:00.075074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:00.170559 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1081/2154] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:33:59 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:00.174623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:00.185521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196e5cc-b1fd-4c38-8947-b90ae233be57 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:00.192743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:00.192743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:00.192743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:00.192743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:00.193175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:00.194030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:00.203736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1074/2155] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:00.207672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:00.218642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea4cb6-1555-40b2-9172-b4f29f9e4032 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:00.229564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713a6ee4-33a3-41f2-aa12-16438a302dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:00.229564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a6ee4-33a3-41f2-aa12-16438a302dda admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:00.229564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a6ee4-33a3-41f2-aa12-16438a302dda admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:00.229564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a6ee4-33a3-41f2-aa12-16438a302dda admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:00.229948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a6ee4-33a3-41f2-aa12-16438a302dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:00.231070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713a6ee4-33a3-41f2-aa12-16438a302dda admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:00.243662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1082/2156] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:00.248029 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:00.259366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42586d22-1ba1-4da0-9566-c3ec46d48c02 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:00.264227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:00.264907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:00.265190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:00.265462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] PATH_INFO: `/v3/projects/e97eb34021d44eb0a99763034c1e9e87/users/7b317f585f8a4da3abe24ab5cf4ee0c0/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:00.284602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97eb34021d44eb0a99763034c1e9e87, user_id=7b317f585f8a4da3abe24ab5cf4ee0c0, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:00.284786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:00.361686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1075/2157] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:00 2022] PUT /identity/v3/projects/e97eb34021d44eb0a99763034c1e9e87/users/7b317f585f8a4da3abe24ab5cf4ee0c0/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:00.367450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:00.367768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:00.368044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:00.387505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:00.388718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:00.410690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:00.456448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1083/2158] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:00.466772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:00.468492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:00.527681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c0f04-b72d-4aae-9feb-af83beae58a1 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:00.531909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:00.532634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:00.532938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:00.533214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:00.543801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:00.545200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:00.555509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1076/2159] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:02.218800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:02.220491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:02.301190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d88c2-c711-4bba-a61d-1e7cb396f742 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:02.305571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:02.306195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:02.306477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:02.306752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:02.308638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:02.383941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:02.386806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e91d30-a38c-4a13-9e35-2e3c7d8ed36d service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:02.398232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1084/2160] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:06.367488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:06.395340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bfda9-92da-4ef4-b598-2651ff1eea43 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:06.395340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:06.395340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:06.395340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:06.395956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:06.403071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:06.513461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:06.524986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:06.528212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:06.542392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1077/2161] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:06.608780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce0093-ec87-441c-8ba9-16dfa9108f59 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:06.609279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce0093-ec87-441c-8ba9-16dfa9108f59 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:06.609379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce0093-ec87-441c-8ba9-16dfa9108f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:06.640961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-41ce0093-ec87-441c-8ba9-16dfa9108f59 None None] MFA Rules not processed for user `7b317f585f8a4da3abe24ab5cf4ee0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:06.642428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-41ce0093-ec87-441c-8ba9-16dfa9108f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:06.746097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1085/2162] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:06.756513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:06.760493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:06.875215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef558e4-f700-473f-a47d-7dd4e5b34d7c service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:06.885780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:06.891274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:06.891274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:06.891274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:06.909888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:06.911740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:06.929962 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1078/2163] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:09.973392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019a0f-df3e-4623-9a1c-6b46eb598b24 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:09.973865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019a0f-df3e-4623-9a1c-6b46eb598b24 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:09.973865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba019a0f-df3e-4623-9a1c-6b46eb598b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:09.996129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ba019a0f-df3e-4623-9a1c-6b46eb598b24 None None] MFA Rules not processed for user `7b317f585f8a4da3abe24ab5cf4ee0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:09.997811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ba019a0f-df3e-4623-9a1c-6b46eb598b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:10.006827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1086/2164] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:13.666965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:13.668490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:13.681816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b4fd7b-3955-40ac-8780-29b290fa53bc service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:13.686774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:13.687379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:13.687665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:13.687943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] PATH_INFO: `/v3/users/841bc9717d6247888cda5118d5e490ed` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:13.689197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=841bc9717d6247888cda5118d5e490ed)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:13.691147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:13.927102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841bc9717d6247888cda5118d5e490ed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:13.939262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841bc9717d6247888cda5118d5e490ed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:13.940079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841bc9717d6247888cda5118d5e490ed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:13.948435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841bc9717d6247888cda5118d5e490ed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:14.040977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841bc9717d6247888cda5118d5e490ed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:14.050440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1079/2165] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:13 2022] DELETE /identity/v3/users/841bc9717d6247888cda5118d5e490ed => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:14.055097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:14.055599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:14.055698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:14.063371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:14.086703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:14.087996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:14.210361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1087/2166] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:14.222701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:14.224166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:14.302695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c22167-0c8b-42ea-afde-111df9a110aa admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:14.309688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:14.311717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:14.311717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:14.311717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:14.322631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:14.324468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:14.325157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:14.342731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1080/2167] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:15.028854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:15.030894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:15.036051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:15.037503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:15.057220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38385b35-b654-4cdd-94da-591ebcf2d54b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:15.057726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c34de-7074-41c5-8279-f2d751c711e2 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:15.066340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:15.073928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:15.076032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:15.076324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:15.076542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] PATH_INFO: `/v3/projects/1ba0b09aa1bc4f898ae9e97f6b87f5f7` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:15.077773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba0b09aa1bc4f898ae9e97f6b87f5f7)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:15.078747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:15.079080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:15.079384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] PATH_INFO: `/v3/users/b200e51e32204cd58dd8c9561f30867f` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:15.081290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:15.081429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b200e51e32204cd58dd8c9561f30867f)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:15.083850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:15.370375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b200e51e32204cd58dd8c9561f30867f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.388783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba0b09aa1bc4f898ae9e97f6b87f5f7', 'request_id': 'req-03f0bcfa-1949-4b48-bc21-89854ab71602'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.389497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b200e51e32204cd58dd8c9561f30867f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.390982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b200e51e32204cd58dd8c9561f30867f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.406305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba0b09aa1bc4f898ae9e97f6b87f5f7', 'request_id': 'req-03f0bcfa-1949-4b48-bc21-89854ab71602'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.415192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b200e51e32204cd58dd8c9561f30867f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.418768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba0b09aa1bc4f898ae9e97f6b87f5f7', 'request_id': 'req-03f0bcfa-1949-4b48-bc21-89854ab71602'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.555601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b69ded71-fa50-40bb-917e-1dcefde718cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b200e51e32204cd58dd8c9561f30867f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.568141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1081/2168] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:15 2022] DELETE /identity/v3/users/b200e51e32204cd58dd8c9561f30867f => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:15.569628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:15.569913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:15.570190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:15.605744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:15.607398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:15.625249 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] The token cache is being invalidate because project 1ba0b09aa1bc4f898ae9e97f6b87f5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:15.625695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.632544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:15.632544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1088/2169] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:15 2022] DELETE /identity/v3/projects/1ba0b09aa1bc4f898ae9e97f6b87f5f7 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:15.643488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:15.653964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:15.741715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1082/2170] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:15.751649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:15.756967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:15.840991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0bcfa-1949-4b48-bc21-89854ab71602 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:15.850870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:15.851895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:15.852268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:15.852771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] PATH_INFO: `/v3/users/68ba323ca9454d28a815d18267021905` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:15.854283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ba323ca9454d28a815d18267021905)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:15.856523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:15.867653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d4e5c-4021-4e67-a645-6a7b8433da72 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:15.872053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:15.873061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:15.873194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:15.873378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:15.888036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:15.891409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:15.909378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1083/2171] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:15.990807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:16.103328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ba323ca9454d28a815d18267021905'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.120547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ba323ca9454d28a815d18267021905'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.120547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ba323ca9454d28a815d18267021905'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.125490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ba323ca9454d28a815d18267021905'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.186844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ba323ca9454d28a815d18267021905'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.194752 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1089/2172] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:15 2022] DELETE /identity/v3/users/68ba323ca9454d28a815d18267021905 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:16.453928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:16.456529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:16.470770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bef0b-22ac-4b77-9301-d677721a7436 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:16.497118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:16.561648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:16.562529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:16.562922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:16.563436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] PATH_INFO: `/v3/projects/82abb82e7ee24c03ac62c14690857553` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:16.564648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82abb82e7ee24c03ac62c14690857553)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:16.566997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:16.704402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82abb82e7ee24c03ac62c14690857553', 'request_id': 'req-2841705d-2034-457b-817f-a9ae40f64e92'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.714218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82abb82e7ee24c03ac62c14690857553', 'request_id': 'req-2841705d-2034-457b-817f-a9ae40f64e92'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.726820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82abb82e7ee24c03ac62c14690857553', 'request_id': 'req-2841705d-2034-457b-817f-a9ae40f64e92'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.818751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] The token cache is being invalidate because project 82abb82e7ee24c03ac62c14690857553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:16.819133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.819822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:16.821392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1084/2173] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:16 2022] DELETE /identity/v3/projects/82abb82e7ee24c03ac62c14690857553 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:16.827515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:16.830602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:16.850230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:16.850527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edcce7-a3ad-4db7-897a-dacb2de26302 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:16.852196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:16.913033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:16.913890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:16.914172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:16.914445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] PATH_INFO: `/v3/users/9d89903c76ef421fbda8774b63ffa621` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:16.915635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d89903c76ef421fbda8774b63ffa621)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:16.917551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:16.926686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841705d-2034-457b-817f-a9ae40f64e92 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:16.931478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:16.932127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:16.932410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:16.932679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] PATH_INFO: `/v3/users/4aca6f188d2f40e4a816c0c148fdbef0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:16.933818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aca6f188d2f40e4a816c0c148fdbef0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:16.935766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:17.139327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d89903c76ef421fbda8774b63ffa621'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.152050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d89903c76ef421fbda8774b63ffa621'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.152786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d89903c76ef421fbda8774b63ffa621'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.160587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d89903c76ef421fbda8774b63ffa621'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.168961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aca6f188d2f40e4a816c0c148fdbef0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.181138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aca6f188d2f40e4a816c0c148fdbef0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.181826 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aca6f188d2f40e4a816c0c148fdbef0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.188493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aca6f188d2f40e4a816c0c148fdbef0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.269638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d89903c76ef421fbda8774b63ffa621'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.276378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1090/2174] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:16 2022] DELETE /identity/v3/users/9d89903c76ef421fbda8774b63ffa621 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:17.277311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:17.278690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:17.308730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:17.326813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-250c2359-5490-47cb-8863-d28704c9b3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aca6f188d2f40e4a816c0c148fdbef0'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.334848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1085/2175] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:16 2022] DELETE /identity/v3/users/4aca6f188d2f40e4a816c0c148fdbef0 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:17.340640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:17.341036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:17.341680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:17.350679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb36c0c-1711-44eb-9431-7860a4004bb9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:17.368160 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:17.369427 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:17.421981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:17.422620 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:17.422902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:17.423171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] PATH_INFO: `/v3/projects/5a4d850b78324fdda1d1027e525cf2f3` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:17.424674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4d850b78324fdda1d1027e525cf2f3)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:17.426703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:17.433368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1086/2176] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:17.444135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:17.445680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:17.504740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01faa9a3-d127-4630-86ac-ace2df6438bc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:17.508483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:17.509096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:17.513231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:17.513231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:17.520405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:17.521853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:17.534954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1087/2177] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:34:17.591557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4d850b78324fdda1d1027e525cf2f3', 'request_id': 'req-913c60c4-aae0-48c7-9bfd-d185a02047ae'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.598836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4d850b78324fdda1d1027e525cf2f3', 'request_id': 'req-913c60c4-aae0-48c7-9bfd-d185a02047ae'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.605905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4d850b78324fdda1d1027e525cf2f3', 'request_id': 'req-913c60c4-aae0-48c7-9bfd-d185a02047ae'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.613092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:17.723524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] The token cache is being invalidate because project 5a4d850b78324fdda1d1027e525cf2f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:17.723524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.723524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:17.723524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1091/2178] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:17 2022] DELETE /identity/v3/projects/5a4d850b78324fdda1d1027e525cf2f3 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:17.725399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:17.727002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:17.750285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565a7b6-472a-4b0c-89a0-055494b0e7c0 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:17.833486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:17.837175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:17.837175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:17.837175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] PATH_INFO: `/v3/users/a1e817cb0b154cb7b740f8a92fcbc144` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:17.841633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1e817cb0b154cb7b740f8a92fcbc144)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:17.844614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:17.914371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:17.916025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:17.932422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c60c4-aae0-48c7-9bfd-d185a02047ae admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:17.938821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:17.940233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:17.940664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:17.941082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] PATH_INFO: `/v3/projects/84114c87f813436db493d36ab2539c30` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:17.942570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=84114c87f813436db493d36ab2539c30)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:17.944745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.071401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1e817cb0b154cb7b740f8a92fcbc144'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.086494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1e817cb0b154cb7b740f8a92fcbc144'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.087680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1e817cb0b154cb7b740f8a92fcbc144'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.098026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1e817cb0b154cb7b740f8a92fcbc144'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.114346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84114c87f813436db493d36ab2539c30', 'request_id': 'req-92220803-92e2-4e52-838f-fe3fb5b32dec'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.121570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84114c87f813436db493d36ab2539c30', 'request_id': 'req-92220803-92e2-4e52-838f-fe3fb5b32dec'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.131621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84114c87f813436db493d36ab2539c30', 'request_id': 'req-92220803-92e2-4e52-838f-fe3fb5b32dec'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.209135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1e817cb0b154cb7b740f8a92fcbc144'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.216023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1088/2179] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:17 2022] DELETE /identity/v3/users/a1e817cb0b154cb7b740f8a92fcbc144 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:18.221981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:18.221981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:18.288581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ef4a0-5262-4ad6-a2e6-e7406ca0f683 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:18.293232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:18.293867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.294141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.294412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] PATH_INFO: `/v3/projects/f1b7329e2ca84e198d1c1e5869286a9f` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.295577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b7329e2ca84e198d1c1e5869286a9f)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:18.297565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.311865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] The token cache is being invalidate because project 84114c87f813436db493d36ab2539c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:18.312810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.313718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-92220803-92e2-4e52-838f-fe3fb5b32dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.316987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1092/2180] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:17 2022] DELETE /identity/v3/projects/84114c87f813436db493d36ab2539c30 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:18.321414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181ba3-ca1d-455d-acb6-6c8f0c3862c0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.321872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181ba3-ca1d-455d-acb6-6c8f0c3862c0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.322308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181ba3-ca1d-455d-acb6-6c8f0c3862c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.344898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-77181ba3-ca1d-455d-acb6-6c8f0c3862c0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:18.346238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-77181ba3-ca1d-455d-acb6-6c8f0c3862c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:18.357675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1093/2181] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:18.366089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.366363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.366633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.388990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:18.390380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:18.398766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1094/2182] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:18.403241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:18.415161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e99932-2c17-4c55-b8f3-03935f710ddc None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:18.428672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b7329e2ca84e198d1c1e5869286a9f', 'request_id': 'req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.438409 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b7329e2ca84e198d1c1e5869286a9f', 'request_id': 'req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.445761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b7329e2ca84e198d1c1e5869286a9f', 'request_id': 'req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.504993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:18.505756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.508573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.508573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.508573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:18.508573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:18.509729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.522839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1095/2183] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:34:18.526929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:18.538171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef01158-a934-48ee-9feb-ed2324333492 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:18.543687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:18.544691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.545184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.545698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.546454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:18.548987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.559919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] The token cache is being invalidate because project f1b7329e2ca84e198d1c1e5869286a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:18.560414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.561138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.563485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1089/2184] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:18 2022] DELETE /identity/v3/projects/f1b7329e2ca84e198d1c1e5869286a9f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:18.567292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:18.568936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:18.600421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8645fa3-4daa-4d7d-8caa-22a857d14d6c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:18.606890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:18.608708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.608708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.609019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] PATH_INFO: `/v3/users/8cb878467ce44140a43e8846336876fa` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.613266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb878467ce44140a43e8846336876fa)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:18.617147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.661619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1096/2185] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:18 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:18.665775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:18.668032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:18.744655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab46c2d-f748-4d81-99a5-4fbbbaf814da admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:18.750498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:18.751188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.751578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.751912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.753650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:18.755817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.904081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1097/2186] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:18 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:18.905616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:18.908074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:18.925098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e179f-dac4-40d3-96e8-63e47987f3da admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:18.932557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:18.933221 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:18.933506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:18.933844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] PATH_INFO: `/v3/projects/ebb6e86c1a3e413ba95bc0b339b5f6e5` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:18.936803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb6e86c1a3e413ba95bc0b339b5f6e5)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:18.937726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:18.941765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb878467ce44140a43e8846336876fa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.957400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb878467ce44140a43e8846336876fa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.958159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb878467ce44140a43e8846336876fa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:18.965310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb878467ce44140a43e8846336876fa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.042952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:19.043256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb878467ce44140a43e8846336876fa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.053006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1090/2187] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:18 2022] DELETE /identity/v3/users/8cb878467ce44140a43e8846336876fa => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:19.056769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.060338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.069700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb6e86c1a3e413ba95bc0b339b5f6e5', 'request_id': 'req-d92ef521-4ca9-480b-b7f5-218050fc98e9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.086452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb6e86c1a3e413ba95bc0b339b5f6e5', 'request_id': 'req-d92ef521-4ca9-480b-b7f5-218050fc98e9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.104705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb6e86c1a3e413ba95bc0b339b5f6e5', 'request_id': 'req-d92ef521-4ca9-480b-b7f5-218050fc98e9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.161596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22d89-00d6-4117-9e74-25e12b4904de admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.205219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] The token cache is being invalidate because project ebb6e86c1a3e413ba95bc0b339b5f6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:19.205219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.205219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d92ef521-4ca9-480b-b7f5-218050fc98e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:19.206021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1098/2188] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:18 2022] DELETE /identity/v3/projects/ebb6e86c1a3e413ba95bc0b339b5f6e5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:19.211845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.212253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.212873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.260458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:19.262106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.296845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.298226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.298862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.299383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.300965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.303670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.322386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1091/2189] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:19.327292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.330458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.354552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14897bea-10e6-4998-92d6-5a085e5b5ebd admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.364980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf44ae-4251-4f2a-963a-6bd60cf8877b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.365671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf44ae-4251-4f2a-963a-6bd60cf8877b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.366572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf44ae-4251-4f2a-963a-6bd60cf8877b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.367122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf44ae-4251-4f2a-963a-6bd60cf8877b admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.367819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf44ae-4251-4f2a-963a-6bd60cf8877b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.370984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbf44ae-4251-4f2a-963a-6bd60cf8877b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.383180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1099/2190] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:19.384058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1092/2191] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:19.388582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.389739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.390395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.390676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.400343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788ace0-7d09-46b9-a86c-5a4b0eabdd9f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.405709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bef96d-e2d2-4eda-aa0e-8794f58a41dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.410629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bef96d-e2d2-4eda-aa0e-8794f58a41dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.410629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bef96d-e2d2-4eda-aa0e-8794f58a41dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.410629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bef96d-e2d2-4eda-aa0e-8794f58a41dd admin admin] PATH_INFO: `/v3/projects/7e0952b6cdf344ddbca3415afa1fb625/users/e75f503f79754742b0ac9a06061f2837/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.414438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:19.415715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.422454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bef96d-e2d2-4eda-aa0e-8794f58a41dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e0952b6cdf344ddbca3415afa1fb625, user_id=e75f503f79754742b0ac9a06061f2837, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.423704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1093/2192] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:19.425703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bef96d-e2d2-4eda-aa0e-8794f58a41dd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.428556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.437960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc9937-6cf1-479f-b8df-4f2af3745f3a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.443025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.443666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.443945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.444216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.444857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:19.445157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.446821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.458611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1094/2193] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:34:19.463293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.474914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376bb01-13ad-45e0-b018-fa0c04c9fd9d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.480097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.480726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.481012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.481287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.482096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.484119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.518695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1100/2194] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:19 2022] PUT /identity/v3/projects/7e0952b6cdf344ddbca3415afa1fb625/users/e75f503f79754742b0ac9a06061f2837/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:19.523845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.527971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.528387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.562208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:19.562208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.583731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:19.597476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1095/2195] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:19 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:19.602018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.657735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1101/2196] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:19.675513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.677011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.692400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edc9c6-f2ee-46df-8c81-34a1a0e49bd4 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.697571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.698410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.698701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.699081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.699762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.701754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.737062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6372d-a9c7-469a-b799-145afc8bcf72 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.787063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.787700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.787976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.788244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.797196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.798610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.808881 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1102/2197] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:19.809735 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.811265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:19.825722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a64f9-7f60-4270-94cb-e7daad039fe2 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.836690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1096/2198] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:19 2022] POST /identity/v3/users => generated 287 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:19.842702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.846618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.848092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.848693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.849199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] PATH_INFO: `/v3/projects/530b92c81ee249f0884014abbcb50752` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.851672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530b92c81ee249f0884014abbcb50752)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.854230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.857318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3c5cb-f606-40fc-8341-20394cb5e7ea admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.870332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.870958 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.871276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.871535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.872193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.873839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.888751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1097/2199] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:34:19.894054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.907022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ac897-e7cc-49ad-95af-0a2cc1b46a07 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.912086 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.912692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.913388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.913388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.913875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.915483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:19.926368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1098/2200] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:19.930352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:19.943106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ee9e1-ddd3-40ad-b529-0c479d5897df admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:19.946560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6abe0c-3e7e-4b76-aef5-17ce9aa5cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:19.947214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6abe0c-3e7e-4b76-aef5-17ce9aa5cbd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:19.947525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6abe0c-3e7e-4b76-aef5-17ce9aa5cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:19.947878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6abe0c-3e7e-4b76-aef5-17ce9aa5cbd2 admin admin] PATH_INFO: `/v3/projects/d9f184a8af604b1886ea8bb41ec362af/users/1da8a9079db544a9829043fa1f5f6e2a/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:19.964491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6abe0c-3e7e-4b76-aef5-17ce9aa5cbd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f184a8af604b1886ea8bb41ec362af, user_id=1da8a9079db544a9829043fa1f5f6e2a, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:19.967121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6abe0c-3e7e-4b76-aef5-17ce9aa5cbd2 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.112670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1099/2201] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:19 2022] PUT /identity/v3/projects/d9f184a8af604b1886ea8bb41ec362af/users/1da8a9079db544a9829043fa1f5f6e2a/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:20.120226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530b92c81ee249f0884014abbcb50752', 'request_id': 'req-f74b8594-a13e-4537-9398-29a992897f75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:20.123644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.123932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.124233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.125815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530b92c81ee249f0884014abbcb50752', 'request_id': 'req-f74b8594-a13e-4537-9398-29a992897f75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:20.132828 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530b92c81ee249f0884014abbcb50752', 'request_id': 'req-f74b8594-a13e-4537-9398-29a992897f75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:20.158997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] MFA Rules not processed for user `1da8a9079db544a9829043fa1f5f6e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:20.160985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:20.231873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1100/2202] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:20.243877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] The token cache is being invalidate because project 530b92c81ee249f0884014abbcb50752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:20.243877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:20.244788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f74b8594-a13e-4537-9398-29a992897f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:20.244905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.246633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1103/2203] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:19 2022] DELETE /identity/v3/projects/530b92c81ee249f0884014abbcb50752 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:20.247684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:20.251277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938ce6-3c0e-40de-afb6-1eaced5f7203 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.251787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938ce6-3c0e-40de-afb6-1eaced5f7203 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.253030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe938ce6-3c0e-40de-afb6-1eaced5f7203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.276994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-fe938ce6-3c0e-40de-afb6-1eaced5f7203 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:20.278209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fe938ce6-3c0e-40de-afb6-1eaced5f7203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:20.317159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a8113-6cbb-4e46-b682-e7098aac496f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.325193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.325817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.326094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.326365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.328152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:20.339475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.341132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.341679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1104/2204] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:20.352455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.354509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.354509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.356477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1101/2205] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:34:20.376681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:20.378058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:20.387626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1105/2206] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:20.391794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.404790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c5e52-0c06-48ae-b782-f400eaf1401a service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.411749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.412879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.413262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.413675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.414442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:20.414899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.417030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.440855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1102/2207] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:34:20.443726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.460350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e14be2-438a-46b4-a4e8-b90738979f9e None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.467767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.469269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.469770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.470260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.472823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.475501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.591182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1106/2208] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:20 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:20.595831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.605750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee0ed6-b7c1-469c-89d7-41565096796a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.673755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.674948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.675249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.675797 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.676476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.678438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.818156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1103/2209] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:20 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:20.822464 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.832928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a571ce-dedb-4163-9622-331ebb670bd1 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.837865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.838460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.838740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.839010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.839761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.841335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.852084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1107/2210] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:20.856789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.874383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3e7ff-9a57-4075-8abd-23fc6f498aa0 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.883560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94545ae-53ef-4871-bef1-7f83ce33153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.884177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94545ae-53ef-4871-bef1-7f83ce33153e admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.884459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94545ae-53ef-4871-bef1-7f83ce33153e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.884734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94545ae-53ef-4871-bef1-7f83ce33153e admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.885384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94545ae-53ef-4871-bef1-7f83ce33153e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.890931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94545ae-53ef-4871-bef1-7f83ce33153e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:20.907298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1104/2211] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:20.909132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:20.920074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e430f51-7fd8-46b9-bc05-606f8b61c0bb admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:20.926739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:20.928200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:20.928372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:20.928732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] PATH_INFO: `/v3/projects/fb6397d5bb724205bc6b838b00553997/users/6d632b57fb274f59b3c3cfbdd9348602/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:20.948934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb6397d5bb724205bc6b838b00553997, user_id=6d632b57fb274f59b3c3cfbdd9348602, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:20.952961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.045685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1108/2212] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:20 2022] PUT /identity/v3/projects/fb6397d5bb724205bc6b838b00553997/users/6d632b57fb274f59b3c3cfbdd9348602/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:21.051051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541918b-7b4e-484a-8596-8c7efd0d3282 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.051403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541918b-7b4e-484a-8596-8c7efd0d3282 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.051680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541918b-7b4e-484a-8596-8c7efd0d3282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.082883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2541918b-7b4e-484a-8596-8c7efd0d3282 None None] MFA Rules not processed for user `6d632b57fb274f59b3c3cfbdd9348602`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:21.084178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2541918b-7b4e-484a-8596-8c7efd0d3282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:21.171809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1105/2213] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:21.176406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.240416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5047-f2db-4f24-a957-31625cce30c4 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.245818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.246471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.246749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.247020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.247729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.249615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.252754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.253117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.253511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.287661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] MFA Rules not processed for user `e75f503f79754742b0ac9a06061f2837`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:21.290396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:21.391268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1109/2214] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:21 2022] POST /identity/v3/projects => generated 367 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:21.393573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.397171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1106/2215] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:21.402448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.403846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:21.426215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c983ec-a151-4869-b379-3d14785afbdb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.502754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12ae40-045d-4106-874f-45ff98255c2e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.502891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.502891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.502891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.502891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.503183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.506870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.507685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.508402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.508678 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.509159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.509801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.512195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.622273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1110/2216] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:21 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:21.622978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1107/2217] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:21 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:21.624976 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.625072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.635314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53981fa8-8805-4f7e-bb11-6a05c57c8f52 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.714434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258a9c3-1e0e-442e-ace6-884e4d2b9a42 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.719660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.720288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.721108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.721574 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.722279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.724277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.726515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.729556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.729556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.729556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.729556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.730067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.746026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1111/2218] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:34:21.747135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.754204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:21.764167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cac905-2beb-4b04-b75f-27b1a1dcbb9e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.774098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.775190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.775628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.776026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] PATH_INFO: `/v3/projects/8016b88a29af4092a9d66867a8f8a35a/users/4212bd56f05d431bb4e1442e9b982dee/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.794521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8016b88a29af4092a9d66867a8f8a35a, user_id=4212bd56f05d431bb4e1442e9b982dee, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.797820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.807537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1108/2219] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:21 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:21.808609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.819499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558b9c3-799b-4d9f-a081-4053f850f1b5 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.824737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.825383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.825659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.825935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.826584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.828235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.840033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1109/2220] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:21.844013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.854948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d1272-4559-4e5a-8fdc-220140efe00d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.863049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e387969-d302-4072-9802-669073872158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.863941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e387969-d302-4072-9802-669073872158 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.864460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e387969-d302-4072-9802-669073872158 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.865964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e387969-d302-4072-9802-669073872158 admin admin] PATH_INFO: `/v3/projects/1df9b2b830bb4b46ad1c1b7b0a60af8a/users/29b85a44a714445685828bf50632cc51/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:21.900611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e387969-d302-4072-9802-669073872158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df9b2b830bb4b46ad1c1b7b0a60af8a, user_id=29b85a44a714445685828bf50632cc51, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:21.903562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e387969-d302-4072-9802-669073872158 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:21.924652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1112/2221] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:21 2022] PUT /identity/v3/projects/8016b88a29af4092a9d66867a8f8a35a/users/4212bd56f05d431bb4e1442e9b982dee/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:21.928617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:21.993821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144d213-8679-4abb-a80f-f31565cfc350 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:21.998694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:21.999349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:21.999631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:21.999907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.000591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:22.002885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:22.016909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1113/2222] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:34:22.018327 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:22.022332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1110/2223] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:21 2022] PUT /identity/v3/projects/1df9b2b830bb4b46ad1c1b7b0a60af8a/users/29b85a44a714445685828bf50632cc51/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:22.027282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e387969-d302-4072-9802-669073872158 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:22.030002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac55014-c5f5-41c1-895a-507ba4bb9706 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:22.115334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e387969-d302-4072-9802-669073872158 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:22.119697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e560a7e-dbe4-4416-8cbc-44f934aae9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:22.120510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e560a7e-dbe4-4416-8cbc-44f934aae9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:22.120649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e560a7e-dbe4-4416-8cbc-44f934aae9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:22.120940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e560a7e-dbe4-4416-8cbc-44f934aae9c9 admin admin] PATH_INFO: `/v3/projects/8016b88a29af4092a9d66867a8f8a35a/users/4212bd56f05d431bb4e1442e9b982dee/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.123160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e560a7e-dbe4-4416-8cbc-44f934aae9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8016b88a29af4092a9d66867a8f8a35a, user_id=4212bd56f05d431bb4e1442e9b982dee, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:22.127038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:22.127816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:22.128952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:22.128952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.129217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:22.132530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e560a7e-dbe4-4416-8cbc-44f934aae9c9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:22.133818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:22.147010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1111/2224] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:22.150727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:22.156447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1114/2225] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:22 2022] PUT /identity/v3/projects/8016b88a29af4092a9d66867a8f8a35a/users/4212bd56f05d431bb4e1442e9b982dee/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:22.167600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:22.167894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:22.168172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.174147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b86f97-9639-4c2b-a9f2-68805179a115 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:22.196496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] MFA Rules not processed for user `4212bd56f05d431bb4e1442e9b982dee`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:22.197716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:22.246740 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:22.247865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:22.248179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:22.248407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] PATH_INFO: `/v3/projects/1df9b2b830bb4b46ad1c1b7b0a60af8a/users/29b85a44a714445685828bf50632cc51/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.250365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df9b2b830bb4b46ad1c1b7b0a60af8a, user_id=29b85a44a714445685828bf50632cc51, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:22.253857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:22.264001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1112/2226] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:22 2022] PUT /identity/v3/projects/1df9b2b830bb4b46ad1c1b7b0a60af8a/users/29b85a44a714445685828bf50632cc51/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:22.273884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1115/2227] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:22.286675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:22.288346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:22.353419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563f503-e6ae-4bc1-be07-c9b71480ee8b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:22.357818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c158-4111-43ee-a692-a5752b2333d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:22.358566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c158-4111-43ee-a692-a5752b2333d1 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:22.358858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c158-4111-43ee-a692-a5752b2333d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:22.359132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c158-4111-43ee-a692-a5752b2333d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.438084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233c158-4111-43ee-a692-a5752b2333d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:22.445503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2233c158-4111-43ee-a692-a5752b2333d1 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:22.464828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1113/2228] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:22.533183 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:22.706578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:22.806299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe7b96-a5a8-4965-b1c3-e7ea4333e96c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:22.810140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:22.810709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:22.810987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:22.811281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:22.883172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:22.885103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:22.897196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1116/2229] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:23.903513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f5052-b0a1-4464-8674-fe280d64adae None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:23.904040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f5052-b0a1-4464-8674-fe280d64adae None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:23.904040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f5052-b0a1-4464-8674-fe280d64adae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:23.944242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-6a2f5052-b0a1-4464-8674-fe280d64adae None None] MFA Rules not processed for user `29b85a44a714445685828bf50632cc51`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:23.946184 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6a2f5052-b0a1-4464-8674-fe280d64adae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:24.036109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1114/2230] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:24.048360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:24.057400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfb931-89e8-4b55-97b9-2a1b629d0189 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:24.061289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:24.061885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:24.062157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:24.062426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:24.103581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:24.103980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:24.104269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:24.123555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] MFA Rules not processed for user `c0aa06eea99f4011ab85ec51b0e60f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:24.124747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:24.127601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:24.129086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:24.138130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1117/2231] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:24.184811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1115/2232] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:25.723264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:25.973341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:25.974822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:25.987268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfb8e3-4ae0-4cef-9439-01907292d316 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:25.990601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d1b4-48a5-4553-b361-780676c071a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:25.991195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d1b4-48a5-4553-b361-780676c071a8 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:25.991608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d1b4-48a5-4553-b361-780676c071a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:25.991767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d1b4-48a5-4553-b361-780676c071a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:26.001043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d1b4-48a5-4553-b361-780676c071a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:26.002854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d1b4-48a5-4553-b361-780676c071a8 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:26.013385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1118/2233] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:34:26.417794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:28.096483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:28.338244 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:29.640792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:29.840833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:30.466671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:30.532252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:31.372147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:31.499079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:34.207541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:34.257098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:34.273141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa6bb4-d8b6-4a6f-903a-3ef5af8aa674 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:34.278276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:34.278920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:34.279198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:34.279576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] PATH_INFO: `/v3/users/6d632b57fb274f59b3c3cfbdd9348602` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:34.280757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d632b57fb274f59b3c3cfbdd9348602)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:34.282898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:34.483643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d632b57fb274f59b3c3cfbdd9348602'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:34.497677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d632b57fb274f59b3c3cfbdd9348602'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:34.498746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d632b57fb274f59b3c3cfbdd9348602'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:34.508578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d632b57fb274f59b3c3cfbdd9348602'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:34.625587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d632b57fb274f59b3c3cfbdd9348602'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:34.633005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1116/2234] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:34 2022] DELETE /identity/v3/users/6d632b57fb274f59b3c3cfbdd9348602 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:34.637715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:34.638640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:34.638640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:34.657978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:34.659514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:34.720141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1119/2235] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:34.729928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:34.731379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:34.791483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d67a8-34db-4839-acc0-fc72793360e1 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:34.795560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:34.796790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:34.796790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:34.796790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:34.812682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:34.814682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:34.829556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1117/2236] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:35.351215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:35.353950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:35.373023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d706873-8398-494a-8e02-12389a7f78f3 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:35.381865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:35.382596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:35.382902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:35.383372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] PATH_INFO: `/v3/projects/fb6397d5bb724205bc6b838b00553997` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:35.384957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb6397d5bb724205bc6b838b00553997)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:35.387395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:35.502772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb6397d5bb724205bc6b838b00553997', 'request_id': 'req-603ca7a4-7a19-4dd1-87dc-5b62b2052545'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.508599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb6397d5bb724205bc6b838b00553997', 'request_id': 'req-603ca7a4-7a19-4dd1-87dc-5b62b2052545'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.514636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb6397d5bb724205bc6b838b00553997', 'request_id': 'req-603ca7a4-7a19-4dd1-87dc-5b62b2052545'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.621151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] The token cache is being invalidate because project fb6397d5bb724205bc6b838b00553997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:35.621562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.622360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.624736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1120/2237] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:35 2022] DELETE /identity/v3/projects/fb6397d5bb724205bc6b838b00553997 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:35.635594 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:35.646781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fa36f-50a9-4100-b438-0c3261cd4c46 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:35.713991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:35.721691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:35.721691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:35.721691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] PATH_INFO: `/v3/users/4212bd56f05d431bb4e1442e9b982dee` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:35.721691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4212bd56f05d431bb4e1442e9b982dee)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:35.721691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:35.732366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:35.987182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4212bd56f05d431bb4e1442e9b982dee'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.999051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4212bd56f05d431bb4e1442e9b982dee'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:35.999770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4212bd56f05d431bb4e1442e9b982dee'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.007320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4212bd56f05d431bb4e1442e9b982dee'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.105312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4212bd56f05d431bb4e1442e9b982dee'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.112441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1118/2238] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:35 2022] DELETE /identity/v3/users/4212bd56f05d431bb4e1442e9b982dee => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:36.570348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:36.577050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:36.600234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ca7a4-7a19-4dd1-87dc-5b62b2052545 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:36.665787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:36.666438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:36.667094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:36.667556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] PATH_INFO: `/v3/projects/8016b88a29af4092a9d66867a8f8a35a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:36.668734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8016b88a29af4092a9d66867a8f8a35a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:36.670693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:36.700266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:36.816743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:36.817216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8016b88a29af4092a9d66867a8f8a35a', 'request_id': 'req-2c2fc61c-fe30-4db0-b61b-e14f648f9117'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.818429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:36.828445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8016b88a29af4092a9d66867a8f8a35a', 'request_id': 'req-2c2fc61c-fe30-4db0-b61b-e14f648f9117'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.837834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a445208-0770-40ee-ae9e-91f412efe940 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:36.843026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8016b88a29af4092a9d66867a8f8a35a', 'request_id': 'req-2c2fc61c-fe30-4db0-b61b-e14f648f9117'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.916410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:36.917114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:36.917521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:36.917807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] PATH_INFO: `/v3/users/1da8a9079db544a9829043fa1f5f6e2a` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:36.919512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1da8a9079db544a9829043fa1f5f6e2a)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:36.925193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] The token cache is being invalidate because project 8016b88a29af4092a9d66867a8f8a35a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:36.925193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.925193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2c2fc61c-fe30-4db0-b61b-e14f648f9117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:36.925842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:36.925926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1121/2239] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:36 2022] DELETE /identity/v3/projects/8016b88a29af4092a9d66867a8f8a35a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:36.929108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41147f1d-837e-4a56-a7ca-a1d719e3dc4a None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:36.929406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41147f1d-837e-4a56-a7ca-a1d719e3dc4a None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:36.929711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41147f1d-837e-4a56-a7ca-a1d719e3dc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:36.952539 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-41147f1d-837e-4a56-a7ca-a1d719e3dc4a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:36.953778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-41147f1d-837e-4a56-a7ca-a1d719e3dc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:36.965726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1122/2240] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:36.977210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:36.977521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:36.977795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:36.999716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:37.000935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:37.009608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1123/2241] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:37.013852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.023328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3e6ff-8952-45ac-8170-3c0a8b29a466 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.028166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.028745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.029016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.029286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.029947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:37.030242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.031868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.045548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1124/2242] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:34:37.049831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.059674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23583781-24af-4cd3-abd4-e622a37a6941 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.064989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.065619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.065903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.066183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.066803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.068757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.116939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1da8a9079db544a9829043fa1f5f6e2a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:37.129920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1da8a9079db544a9829043fa1f5f6e2a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:37.130628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1da8a9079db544a9829043fa1f5f6e2a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:37.137658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1da8a9079db544a9829043fa1f5f6e2a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:37.157273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1125/2243] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:37 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:37.162035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.202203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-5631a536-43d3-402d-a08f-260745841a9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1da8a9079db544a9829043fa1f5f6e2a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:37.209852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1119/2244] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:36 2022] DELETE /identity/v3/users/1da8a9079db544a9829043fa1f5f6e2a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:37.214808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.215474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.215760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.230645 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:37.236463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc06e10-d83d-4a9a-a499-605907bd475f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.238877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:37.240301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:37.302376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.305549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.305549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.305549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.305549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.306399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.310660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1120/2245] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:37.320709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:37.321597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.323628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:37.382002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0571ea4-7687-44bd-a415-6a88fb3a6430 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.385598 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.386389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.386664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.386935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.397283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.399083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.400011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1126/2246] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:37 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:37.403982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.405579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:37.409605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1121/2247] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:34:37.417553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53a6ad-4693-445f-81c2-d795bc81bc9f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.422293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.422936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.423514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.423687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.424325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.425896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.435688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1127/2248] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:37.440353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.450473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c434bbda-6289-4c1d-863e-86aac7f460cb service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.457416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2d13-28fa-4d1e-a05f-d349a07cb281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.458075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2d13-28fa-4d1e-a05f-d349a07cb281 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.458354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2d13-28fa-4d1e-a05f-d349a07cb281 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.458621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2d13-28fa-4d1e-a05f-d349a07cb281 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.459549 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe2d13-28fa-4d1e-a05f-d349a07cb281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.462361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe2d13-28fa-4d1e-a05f-d349a07cb281 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.473163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1122/2249] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:34:37.477550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.489208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dfeb44-3ef0-46f9-98a3-33c3e194c52a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.495195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.496133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.496472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.496802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] PATH_INFO: `/v3/projects/2ebb60e821cf40b38ec01549c8a5f664/users/a846d9c2f5a64808ad89fa40aa999483/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.515838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebb60e821cf40b38ec01549c8a5f664, user_id=a846d9c2f5a64808ad89fa40aa999483, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.519739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.643293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1128/2250] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:37 2022] PUT /identity/v3/projects/2ebb60e821cf40b38ec01549c8a5f664/users/a846d9c2f5a64808ad89fa40aa999483/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:37.654049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.654561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.655036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.692471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] MFA Rules not processed for user `a846d9c2f5a64808ad89fa40aa999483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:37.693695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:37.764950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1123/2251] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:37.770001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.829533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0900a6-2607-491a-b52d-d6160b593872 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.834702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.835331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.835614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.835885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.836512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.838462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.857165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:37.858448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:37.873014 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d15267-791d-4795-8b8b-b77c28ad7220 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:37.878835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:37.880185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:37.880185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:37.880185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] PATH_INFO: `/v3/projects/d9f184a8af604b1886ea8bb41ec362af` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:37.881196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f184a8af604b1886ea8bb41ec362af)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:37.883109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:37.928342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:37.960800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1129/2252] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:37 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:37.961837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.027076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340017d7-4b54-4ceb-8892-9c8c9038f98a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.032003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.033409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.033691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.033965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.034583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.036551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.065405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f184a8af604b1886ea8bb41ec362af', 'request_id': 'req-1d4cb633-fdf7-4b69-9e06-7f744d333a91'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:38.070769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f184a8af604b1886ea8bb41ec362af', 'request_id': 'req-1d4cb633-fdf7-4b69-9e06-7f744d333a91'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:38.076959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f184a8af604b1886ea8bb41ec362af', 'request_id': 'req-1d4cb633-fdf7-4b69-9e06-7f744d333a91'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:38.151618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1130/2253] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:37 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:38.152809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.164389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5319-5dbc-4fd8-b068-f7dc03e2d550 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.180875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] The token cache is being invalidate because project d9f184a8af604b1886ea8bb41ec362af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:38.181384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:38.182078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1d4cb633-fdf7-4b69-9e06-7f744d333a91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:38.183870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1124/2254] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:37 2022] DELETE /identity/v3/projects/d9f184a8af604b1886ea8bb41ec362af => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:38.188705 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7fbc-899f-4dd6-8091-2459c91c9eda None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.189009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7fbc-899f-4dd6-8091-2459c91c9eda None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.189288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7fbc-899f-4dd6-8091-2459c91c9eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.208463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2a4e7fbc-899f-4dd6-8091-2459c91c9eda None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:38.210001 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2a4e7fbc-899f-4dd6-8091-2459c91c9eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:38.232771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.233376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.233653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.233924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.234553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.236946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.246610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1131/2255] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:38.250698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.254708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:38.267833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a182e2-2ccc-41bf-bb07-e47157f86e2b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.270220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1125/2256] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:38.272798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.273452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.273734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.274013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] PATH_INFO: `/v3/projects/27614eb6d8d448718b4062872bcac414/users/9af57fb40af84f568888569baa970f5b/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.278430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.281837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.281837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.282088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:38.292195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27614eb6d8d448718b4062872bcac414, user_id=9af57fb40af84f568888569baa970f5b, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.294971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.355058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1132/2257] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:38 2022] PUT /identity/v3/projects/27614eb6d8d448718b4062872bcac414/users/9af57fb40af84f568888569baa970f5b/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:38.359087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.435571 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:38.436803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:38.444313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d66122-9935-4ffc-9f5a-2cc178d787e3 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.445451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1126/2258] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:38.449445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.454758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.454877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.454877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.454877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.454877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.454877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.459901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8c12e-8190-48a7-99ab-90042ebe5536 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.467295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1133/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:34:38.467474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.467474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.467474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.467717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.468286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:38.468590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.470392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.470783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.481128 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9151fe0-090f-4c37-9169-e78c01f9c66a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.484284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1127/2260] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:34:38.486890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2911652-9355-4313-8c00-965a43417d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.487683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2911652-9355-4313-8c00-965a43417d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.488031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2911652-9355-4313-8c00-965a43417d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.488423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2911652-9355-4313-8c00-965a43417d6e admin admin] PATH_INFO: `/v3/projects/27614eb6d8d448718b4062872bcac414/users/9af57fb40af84f568888569baa970f5b/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.489444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.490371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2911652-9355-4313-8c00-965a43417d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27614eb6d8d448718b4062872bcac414, user_id=9af57fb40af84f568888569baa970f5b, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.493079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2911652-9355-4313-8c00-965a43417d6e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.502803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e66503-183e-4884-8ac9-bc12013a4979 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.507208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1134/2261] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:38 2022] PUT /identity/v3/projects/27614eb6d8d448718b4062872bcac414/users/9af57fb40af84f568888569baa970f5b/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:38.534325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.534680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.538526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.565148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] MFA Rules not processed for user `9af57fb40af84f568888569baa970f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:38.566386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:38.584829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.585515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.585873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.586211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.587205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.594019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.643166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1135/2262] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:38.654141 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.655700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:38.696849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1128/2263] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:38 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:38.700859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.716188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da302fb-e7e2-43aa-b715-85a7ae9f1b5b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.721269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:38.763801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757c26-6552-452b-bd2a-c81e2ffd0df1 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.765065 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.765679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.765964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.766238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.768113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:38.769341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.769938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.770323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.770592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.771203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.773251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.828884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.831597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.840759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1136/2264] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:34:38.888468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1129/2265] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:38 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:38.894507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.905698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d0b92-4ad0-45a5-8593-0ee7c540f2d6 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.910823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.911455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.911742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.912041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.912652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.914279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.924861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1137/2266] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:38.941542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.951638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85a0bd-467d-4275-a14b-1c2c2c7d29ca admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.956954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93c2d9-57c8-4c93-9d74-05ca5328e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.957606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c2d9-57c8-4c93-9d74-05ca5328e32a admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.957949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c2d9-57c8-4c93-9d74-05ca5328e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.958289 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93c2d9-57c8-4c93-9d74-05ca5328e32a admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:38.958975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93c2d9-57c8-4c93-9d74-05ca5328e32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:38.960858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93c2d9-57c8-4c93-9d74-05ca5328e32a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:38.972092 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1130/2267] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:38.975991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:38.985923 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b03687-20e8-43f0-a61e-d2c6e4f68a3e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:38.990871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:38.991561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:38.991840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:38.992111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] PATH_INFO: `/v3/projects/fc3040045262415cb0fcb9525ba437bc/users/41fecf50555a4effb8aca217296bbad6/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:39.008277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3040045262415cb0fcb9525ba437bc, user_id=41fecf50555a4effb8aca217296bbad6, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:39.011034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:39.077872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:39.084885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1138/2268] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:38 2022] PUT /identity/v3/projects/fc3040045262415cb0fcb9525ba437bc/users/41fecf50555a4effb8aca217296bbad6/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:39.089952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:39.090238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:39.090514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:39.109082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:39.110881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:39.169651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1131/2269] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:39.180201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:39.181996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:39.241840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f67cf-294c-4fd0-96fe-65f469cd5425 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:39.245449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a11e0-db32-4b21-9703-d7ca292d7ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:39.246051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a11e0-db32-4b21-9703-d7ca292d7ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:39.246343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a11e0-db32-4b21-9703-d7ca292d7ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:39.246617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a11e0-db32-4b21-9703-d7ca292d7ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:39.256863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a11e0-db32-4b21-9703-d7ca292d7ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:39.258401 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a11e0-db32-4b21-9703-d7ca292d7ee7 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:39.268931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1139/2270] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:40.320180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:43.450510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:43.461267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c044e8-987d-4b5e-bf91-6ea84b3a6507 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:43.466721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:43.467997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:43.468768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:43.469207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] PATH_INFO: `/v3/users/a846d9c2f5a64808ad89fa40aa999483` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:43.478732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a846d9c2f5a64808ad89fa40aa999483)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:43.480750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:43.680214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a846d9c2f5a64808ad89fa40aa999483'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:43.715639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a846d9c2f5a64808ad89fa40aa999483'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:43.716627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a846d9c2f5a64808ad89fa40aa999483'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:43.728478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a846d9c2f5a64808ad89fa40aa999483'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:43.808441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a846d9c2f5a64808ad89fa40aa999483'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:43.820583 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1132/2271] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:43 2022] DELETE /identity/v3/users/a846d9c2f5a64808ad89fa40aa999483 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:43.825818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:43.826872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:43.827515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:43.870144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:43.874412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:43.995097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1140/2272] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:44.007957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:44.010235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:44.026421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:44.104285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c7987-73e0-4515-bf83-a5895ee1d243 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:44.108758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:44.110958 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:44.112076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:44.112568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:44.125650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:44.127442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:44.138888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1133/2273] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:45.008530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:45.010629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:45.035853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070081df-8e89-4d57-82ff-ca273aa95656 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:45.048350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:45.054898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:45.055494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:45.056531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] PATH_INFO: `/v3/projects/2ebb60e821cf40b38ec01549c8a5f664` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:45.058286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ebb60e821cf40b38ec01549c8a5f664)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:45.060634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:45.215956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ebb60e821cf40b38ec01549c8a5f664', 'request_id': 'req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.239712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ebb60e821cf40b38ec01549c8a5f664', 'request_id': 'req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.249338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ebb60e821cf40b38ec01549c8a5f664', 'request_id': 'req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.405712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] The token cache is being invalidate because project 2ebb60e821cf40b38ec01549c8a5f664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:45.406170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.407973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.410906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1141/2274] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:45 2022] DELETE /identity/v3/projects/2ebb60e821cf40b38ec01549c8a5f664 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:45.415075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:45.431357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557e748-deed-4c5c-9abb-d49f13c847a3 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:45.550753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:45.552989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:45.553278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:45.553673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] PATH_INFO: `/v3/users/9af57fb40af84f568888569baa970f5b` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:45.560489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af57fb40af84f568888569baa970f5b)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:45.563141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:45.863098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af57fb40af84f568888569baa970f5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.873407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:45.875010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:45.888641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af57fb40af84f568888569baa970f5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.889477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af57fb40af84f568888569baa970f5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.893838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42e5f8-26b4-4b7f-a1d1-c8a60c2bdca1 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:45.899977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af57fb40af84f568888569baa970f5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:45.900253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:45.901643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:45.902114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:45.902632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] PATH_INFO: `/v3/users/e97ab3e1e9364cf990d1eb17ee486d06` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:45.904279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97ab3e1e9364cf990d1eb17ee486d06)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:45.908381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:46.029102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-72c8dfa5-0996-4fe9-b984-f538984ba73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af57fb40af84f568888569baa970f5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:46.042256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1134/2275] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:45 2022] DELETE /identity/v3/users/9af57fb40af84f568888569baa970f5b => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:46.132693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97ab3e1e9364cf990d1eb17ee486d06'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:46.161666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97ab3e1e9364cf990d1eb17ee486d06'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:46.162796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97ab3e1e9364cf990d1eb17ee486d06'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:46.173380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:46.176109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97ab3e1e9364cf990d1eb17ee486d06'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:46.277409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97ab3e1e9364cf990d1eb17ee486d06'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:46.289843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1142/2276] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:45 2022] DELETE /identity/v3/users/e97ab3e1e9364cf990d1eb17ee486d06 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:46.291818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:46.292201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:46.292553 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:46.322743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:46.324313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:46.497126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1135/2277] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:46.504831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:46.528565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:46.529698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:46.658798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25268f4-90c4-44ea-be66-394ec2fc5a8c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:46.664855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63e91f-a71e-4c46-9f2f-2efda883799e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:46.666137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63e91f-a71e-4c46-9f2f-2efda883799e service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:46.666639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63e91f-a71e-4c46-9f2f-2efda883799e service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:46.667091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63e91f-a71e-4c46-9f2f-2efda883799e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:46.679762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63e91f-a71e-4c46-9f2f-2efda883799e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:46.681743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63e91f-a71e-4c46-9f2f-2efda883799e service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:46.721154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1143/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:46.845693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:46.847959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:46.880048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3db5b-ef51-4543-91cf-7f106b4382a2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:46.904297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:46.905141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:46.905660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:46.906079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] PATH_INFO: `/v3/projects/27614eb6d8d448718b4062872bcac414` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:46.908279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=27614eb6d8d448718b4062872bcac414)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:46.910654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:47.168241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27614eb6d8d448718b4062872bcac414', 'request_id': 'req-b10af11f-b6d1-4213-88f2-56dd6a683bcd'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.193490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27614eb6d8d448718b4062872bcac414', 'request_id': 'req-b10af11f-b6d1-4213-88f2-56dd6a683bcd'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.217695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27614eb6d8d448718b4062872bcac414', 'request_id': 'req-b10af11f-b6d1-4213-88f2-56dd6a683bcd'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.311543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] The token cache is being invalidate because project 27614eb6d8d448718b4062872bcac414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:47.312088 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.315798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b10af11f-b6d1-4213-88f2-56dd6a683bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.317511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1136/2279] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:46 2022] DELETE /identity/v3/projects/27614eb6d8d448718b4062872bcac414 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:47.323430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:47.324565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:47.324957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:47.378096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:47.378096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:47.493059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1144/2280] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:47.494624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:47.495255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:47.495920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:47.534917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:47.535952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:47.546015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:47.549671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:47.572722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab7a8a-b5fb-4eac-b8e5-b740534a3490 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:47.582170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1137/2281] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:47.584431 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:47.592530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:47.593192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:47.593474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:47.593927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] PATH_INFO: `/v3/projects/aa32873224e04d338c3acb912289781f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:47.600967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa32873224e04d338c3acb912289781f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:47.603032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:47.635166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34186bcb-2462-4368-88b6-14c54a0a8c5c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:47.650127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:47.651125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:47.663166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:47.663760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:47.664375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:47.664756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:47.675156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745464-dc0e-4670-b35e-e3cabef15594 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:47.710462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1138/2282] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:34:47.714538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:47.715051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:47.716211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:47.745187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:47.747616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:47.836540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1139/2283] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:34:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:47.837769 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:47.850910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa32873224e04d338c3acb912289781f', 'request_id': 'req-9a8b416b-3d1f-4c59-a824-7664b7445494'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.861168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436b81e-a0ae-47e4-94c7-ef644a0f2411 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:47.862955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa32873224e04d338c3acb912289781f', 'request_id': 'req-9a8b416b-3d1f-4c59-a824-7664b7445494'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.871831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa32873224e04d338c3acb912289781f', 'request_id': 'req-9a8b416b-3d1f-4c59-a824-7664b7445494'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.945702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbd3b0-2a54-45d3-b330-47f2ffc4429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:47.947941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd3b0-2a54-45d3-b330-47f2ffc4429e admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:47.947941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd3b0-2a54-45d3-b330-47f2ffc4429e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:47.947941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbd3b0-2a54-45d3-b330-47f2ffc4429e admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:47.948202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd3b0-2a54-45d3-b330-47f2ffc4429e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:47.950131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbd3b0-2a54-45d3-b330-47f2ffc4429e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:47.976268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] The token cache is being invalidate because project aa32873224e04d338c3acb912289781f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:47.976788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.978593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:47.980818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1145/2284] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:47 2022] DELETE /identity/v3/projects/aa32873224e04d338c3acb912289781f => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:47.982283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:47.984039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.064657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b416b-3d1f-4c59-a824-7664b7445494 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:48.070842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:48.071528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.071807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.072080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.073867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.079679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1140/2285] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:47 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:48.083954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.083954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.083954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.105309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:48.106589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.148808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:48.150411 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf8c637-8a9f-4a75-b54d-3c0ac8a58dbe service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:48.158820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1146/2286] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:34:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:34:48.160507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.160758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.161099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.180623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1141/2287] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:48.187628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:48.187628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.198572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7227c-60fb-4e7e-862f-fa6b7e473869 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:48.203788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9bfcd4-4f3d-447e-8c93-98aab94f10ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:48.204760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bfcd4-4f3d-447e-8c93-98aab94f10ce admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.205126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bfcd4-4f3d-447e-8c93-98aab94f10ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.205306 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9bfcd4-4f3d-447e-8c93-98aab94f10ce admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.207394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bfcd4-4f3d-447e-8c93-98aab94f10ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:48.208457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9bfcd4-4f3d-447e-8c93-98aab94f10ce admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:48.211595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] MFA Rules not processed for user `41fecf50555a4effb8aca217296bbad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:48.216304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.296400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1147/2288] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:48.298186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:48.299515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.343474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d52b1-0002-4166-9fd7-6aff5ae35a44 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:48.354793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:48.392851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.392851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.392851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] PATH_INFO: `/v3/users/805a8f0541914c4eb06aa0dd20bf8cc6` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.392851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805a8f0541914c4eb06aa0dd20bf8cc6)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:48.392851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:48.441319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1142/2289] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:48 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:48.442802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.443155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.443562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.467175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:48.468556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.483534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1143/2290] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:48.484348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:48.486269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.503646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6952b6-bd2c-46cd-8b1d-beac84e0dbcb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:48.508856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:48.509517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.509805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.510070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.510698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:48.512621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:48.783724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1144/2291] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:48 2022] POST /identity/v3/projects => generated 383 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:48.783724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:48.805290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805a8f0541914c4eb06aa0dd20bf8cc6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:48.834351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805a8f0541914c4eb06aa0dd20bf8cc6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:48.836429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805a8f0541914c4eb06aa0dd20bf8cc6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:48.844140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805a8f0541914c4eb06aa0dd20bf8cc6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:48.920344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805a8f0541914c4eb06aa0dd20bf8cc6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:48.923108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c857e0-d122-44d3-b7b9-7d286a78ac34 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:48.929241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:48.932959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:48.932959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:48.932959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:48.932959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:48.933794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1148/2292] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:48 2022] DELETE /identity/v3/users/805a8f0541914c4eb06aa0dd20bf8cc6 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:48.933940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:48.935386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:48.937063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.952251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1145/2293] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:48.958265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:48.958265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:48.985551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca787186-d169-4770-b190-bbc061e93671 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.069741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:49.080313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e073fba-49b1-43ce-af34-7a0faea94b39 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.148046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.150454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.151124 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.151631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.152537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:49.153061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.154929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8450c-3029-41c1-8d4e-d8bfd5761267 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.183053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1149/2294] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:34:49.190164 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.191866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.192347 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.213562 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.214827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.215128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.215421 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.216077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.218054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.239191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:49.241070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:49.256144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1150/2295] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:49.268687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.270541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:49.299615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b0e42-5826-4eac-af0b-16846d898dff None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.306758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.307438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.307717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.307988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.308697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.315428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.341666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1151/2296] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:49.347926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.359667 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f90e8-593b-4512-9e88-a4a8b8b25393 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.365391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.366414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.366987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.367531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.368545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.370885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.377188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:49.380405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1146/2297] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:48 2022] POST /identity/v3/users => generated 298 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:49.381412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.388500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:49.496621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1152/2298] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:49 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:49.497461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ef145-e7cd-44e0-a4da-63eb023d5180 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.498132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.568334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.569242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.570282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.570715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.612968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8871f-a29c-4b5a-80a7-6224cd97d426 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.626358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.628769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.631253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.631713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] PATH_INFO: `/v3/projects/3eb54fe905044c9e8d77a4e151b08dde/users/fd4d1d06e94d46f885310208e04eb3af/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.654283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.657918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.667257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb54fe905044c9e8d77a4e151b08dde, user_id=fd4d1d06e94d46f885310208e04eb3af, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.670528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.671813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1147/2299] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:49.673669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.687264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9cf9c-3139-4780-b97d-80b0c1444bd1 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.693271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.694225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.694908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.695493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.696351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.698450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.712722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1148/2300] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:49.713624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.739832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b289c4-f2bf-4b16-9faf-57569a488a79 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.743793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97212600-9a2a-420d-bc34-7398b1ce3ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.744500 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212600-9a2a-420d-bc34-7398b1ce3ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.744775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212600-9a2a-420d-bc34-7398b1ce3ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.745287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97212600-9a2a-420d-bc34-7398b1ce3ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.746084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97212600-9a2a-420d-bc34-7398b1ce3ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.748441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97212600-9a2a-420d-bc34-7398b1ce3ce6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.778233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1153/2301] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:49 2022] PUT /identity/v3/projects/3eb54fe905044c9e8d77a4e151b08dde/users/fd4d1d06e94d46f885310208e04eb3af/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:49.785334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:49.897063 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1149/2302] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:49 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:49.899383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.900137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.900572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.909708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1828ff7-49b1-4ae4-98c1-2bd974adb2c7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:49.923902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:49.927569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:49.927569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:49.927569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] PATH_INFO: `/v3/projects/dd8ef69b931748e689f6d2a9cd4aceaa/users/03110f36df2d444fad6ff04bc308be4d/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:49.977143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8ef69b931748e689f6d2a9cd4aceaa, user_id=03110f36df2d444fad6ff04bc308be4d, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:49.992808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:49.998909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] MFA Rules not processed for user `fd4d1d06e94d46f885310208e04eb3af`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:50.012040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:50.215378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1154/2303] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:49 2022] PUT /identity/v3/projects/dd8ef69b931748e689f6d2a9cd4aceaa/users/03110f36df2d444fad6ff04bc308be4d/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:50.215378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:50.242039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1150/2304] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:50.242039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:50.258639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa85cd-6abe-4c85-b68c-5acbae9efd57 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:50.337290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de948a22-dc80-4004-a676-6ee43040a7ba admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:50.348061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:50.349009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:50.349186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:50.349678 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:50.351448 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:50.354723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:50.376547 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:50.380751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:50.381137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:50.381521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:50.385262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:50.385443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1155/2305] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:50.385443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:50.386980 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:50.388931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:50.411853 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1151/2306] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:50.414302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:50.436462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557be9d-a49c-4573-a45c-fb132c520d98 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:50.452173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:50.452886 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:50.453197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:50.453590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:50.454847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:50.456794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:50.491099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1152/2307] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:50.492074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:50.513901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119616e-a259-42a5-a0a5-45cb74496558 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:50.528360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d04a6-8a74-4a9b-8919-b3b57d642d66 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:50.533482 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:50.535434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:50.535434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:50.535657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:50.535782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:50.536012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:50.536441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:50.536864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] PATH_INFO: `/v3/projects/dd8ef69b931748e689f6d2a9cd4aceaa/users/03110f36df2d444fad6ff04bc308be4d/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:50.539686 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd8ef69b931748e689f6d2a9cd4aceaa, user_id=03110f36df2d444fad6ff04bc308be4d, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:50.545909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:50.578942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1153/2308] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:50 2022] PUT /identity/v3/projects/dd8ef69b931748e689f6d2a9cd4aceaa/users/03110f36df2d444fad6ff04bc308be4d/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:50.579977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:50.682543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:50.684292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:50.711104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1156/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:50.717903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f434be-c164-4fc6-81d4-e29f8848855a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:50.726117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7603d5f-7034-4455-87e3-57034db5c2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:50.727601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603d5f-7034-4455-87e3-57034db5c2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:50.727870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603d5f-7034-4455-87e3-57034db5c2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:50.728315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7603d5f-7034-4455-87e3-57034db5c2c2 admin admin] PATH_INFO: `/v3/projects/2039a0e1619d46a8927b63797d34dc5e/users/c2de6d1cc2e4439aa4973174c1e37b75/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:50.735596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:50.776543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7603d5f-7034-4455-87e3-57034db5c2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2039a0e1619d46a8927b63797d34dc5e, user_id=c2de6d1cc2e4439aa4973174c1e37b75, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:50.779254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7603d5f-7034-4455-87e3-57034db5c2c2 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:51.042601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:51.044280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:51.060468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1154/2310] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:50 2022] PUT /identity/v3/projects/2039a0e1619d46a8927b63797d34dc5e/users/c2de6d1cc2e4439aa4973174c1e37b75/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:51.071589 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:51.072282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:51.072726 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:51.076536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3699ec-0f85-4f45-8d1d-aaa3de064b2e service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:51.146722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] MFA Rules not processed for user `c2de6d1cc2e4439aa4973174c1e37b75`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:51.149275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:51.251201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:51.252253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:51.252672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:51.252950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] PATH_INFO: `/v3/projects/61e7409c7b6e4520971aabf4a6fd76e1` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:51.254443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e7409c7b6e4520971aabf4a6fd76e1)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:51.256443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:51.269688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:51.293441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1155/2311] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:51.315279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:51.320873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:51.438301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e7409c7b6e4520971aabf4a6fd76e1', 'request_id': 'req-49dea970-7f24-4df6-b861-810fb188681f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:51.452824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e7409c7b6e4520971aabf4a6fd76e1', 'request_id': 'req-49dea970-7f24-4df6-b861-810fb188681f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:51.460943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7213381-f327-406f-b9cf-a451bf5038f9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:51.468829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e7409c7b6e4520971aabf4a6fd76e1', 'request_id': 'req-49dea970-7f24-4df6-b861-810fb188681f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:51.610857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505494ed-b533-4e93-833c-31a18d3b234f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:51.611610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505494ed-b533-4e93-833c-31a18d3b234f service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:51.611985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505494ed-b533-4e93-833c-31a18d3b234f service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:51.612351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505494ed-b533-4e93-833c-31a18d3b234f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:51.676690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] The token cache is being invalidate because project 61e7409c7b6e4520971aabf4a6fd76e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:51.677170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:51.678272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:51.681434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1157/2312] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:51 2022] DELETE /identity/v3/projects/61e7409c7b6e4520971aabf4a6fd76e1 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:51.685682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:51.690866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:51.857548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505494ed-b533-4e93-833c-31a18d3b234f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:51.860286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505494ed-b533-4e93-833c-31a18d3b234f service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:51.928217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1156/2313] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:51.929918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabc7b-7104-42e4-ba61-a6c35cc78ed4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:51.930208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabc7b-7104-42e4-ba61-a6c35cc78ed4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:51.930537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdabc7b-7104-42e4-ba61-a6c35cc78ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:51.994268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-efdabc7b-7104-42e4-ba61-a6c35cc78ed4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:51.995810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-efdabc7b-7104-42e4-ba61-a6c35cc78ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:52.012870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dea970-7f24-4df6-b861-810fb188681f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.029393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1157/2314] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:52.033689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.033981 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.034322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.047463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.051051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.052428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.053543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] PATH_INFO: `/v3/users/e75f503f79754742b0ac9a06061f2837` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.056789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75f503f79754742b0ac9a06061f2837)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.079732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:52.143992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:52.145159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:52.174048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1158/2315] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:52.180269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:52.192169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902a806-5a51-4a2b-90d1-64791a20f76d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.197139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.197762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.198048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.198322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.199394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:52.199775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.201347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:52.228156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1159/2316] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:34:52.229100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:52.240748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e0077-108d-4e1d-a65c-11879c20cdcd admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.246612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.247305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.247596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.247973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.248616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.251912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:52.345696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75f503f79754742b0ac9a06061f2837'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:52.352195 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1160/2317] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:52 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:52.359060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:52.393380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75f503f79754742b0ac9a06061f2837'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:52.396662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75f503f79754742b0ac9a06061f2837'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:52.402813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75f503f79754742b0ac9a06061f2837'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:52.420487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:52.475341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ada818-82c3-4a3e-a213-89b011c72c11 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.481712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.482406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.483093 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.483551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.485346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.487144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75f503f79754742b0ac9a06061f2837'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:52.492178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:52.522869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1158/2318] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:51 2022] DELETE /identity/v3/users/e75f503f79754742b0ac9a06061f2837 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:52.612541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:52.616797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:52.621233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1161/2319] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:52 2022] POST /identity/v3/users => generated 288 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:52.754668 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e96fe-b0af-4e0a-b76d-70ab60d2eba3 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.760532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.765262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.765262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.765262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.765262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.765262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:52.779851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1159/2320] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:52.784941 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:52.797102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8e1c4-de0a-436f-838d-f2f739b45e2b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.803693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb10a5b-a033-4a16-b6ef-3177b4d6c8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.804819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb10a5b-a033-4a16-b6ef-3177b4d6c8df admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.805381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb10a5b-a033-4a16-b6ef-3177b4d6c8df admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.805695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb10a5b-a033-4a16-b6ef-3177b4d6c8df admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.806383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb10a5b-a033-4a16-b6ef-3177b4d6c8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.808191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb10a5b-a033-4a16-b6ef-3177b4d6c8df admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:52.815276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:52.826599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1162/2321] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:52.831153 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:52.846078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910dfba-83bd-43db-ae37-a418d0cfff53 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:52.861116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:52.861994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:52.862637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:52.862908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] PATH_INFO: `/v3/projects/0fb873acb9dc45d98bb0ccde8ee50a5a/users/9f87028b4b624ebb829a4b61da5f9b9d/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:52.896736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb873acb9dc45d98bb0ccde8ee50a5a, user_id=9f87028b4b624ebb829a4b61da5f9b9d, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:52.902399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:53.051332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1160/2322] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:52 2022] PUT /identity/v3/projects/0fb873acb9dc45d98bb0ccde8ee50a5a/users/9f87028b4b624ebb829a4b61da5f9b9d/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:53.061525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:53.062050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:53.062476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:53.121514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] MFA Rules not processed for user `9f87028b4b624ebb829a4b61da5f9b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:53.123203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:53.252009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1163/2323] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:53.263274 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:53.265085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:53.329960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:53.332214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:53.416753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e5264-6165-4a54-9477-ea3c67be3fe1 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:53.425316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:53.426079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:53.426462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:53.426833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:53.452840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:53.459338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:53.518873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1161/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:53.550501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c5c33-8782-4391-8662-8e0974512463 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:53.570284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:53.590841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:53.590841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:53.590841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] PATH_INFO: `/v3/projects/7e0952b6cdf344ddbca3415afa1fb625` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:53.590841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e0952b6cdf344ddbca3415afa1fb625)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:53.590841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:53.815207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e0952b6cdf344ddbca3415afa1fb625', 'request_id': 'req-c655c329-45a5-4130-b248-8b17be5bab6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:53.827714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e0952b6cdf344ddbca3415afa1fb625', 'request_id': 'req-c655c329-45a5-4130-b248-8b17be5bab6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:53.836265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e0952b6cdf344ddbca3415afa1fb625', 'request_id': 'req-c655c329-45a5-4130-b248-8b17be5bab6c'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.073246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] The token cache is being invalidate because project 7e0952b6cdf344ddbca3415afa1fb625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:54.073246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.075259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.084869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:54.087624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1164/2325] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:53 2022] DELETE /identity/v3/projects/7e0952b6cdf344ddbca3415afa1fb625 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:54.100981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc737f-a934-4995-bff5-a5db05e637ff service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:54.209611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:54.210681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:54.211621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:54.211621 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] PATH_INFO: `/v3/users/29b85a44a714445685828bf50632cc51` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:54.212768 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b85a44a714445685828bf50632cc51)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:54.215127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:54.543566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b85a44a714445685828bf50632cc51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.590180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b85a44a714445685828bf50632cc51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.598100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b85a44a714445685828bf50632cc51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.598352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:54.598352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:54.604967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b85a44a714445685828bf50632cc51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.615439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655c329-45a5-4130-b248-8b17be5bab6c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:54.634154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:54.635407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:54.635870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:54.636328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] PATH_INFO: `/v3/users/fd4d1d06e94d46f885310208e04eb3af` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:54.637955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4d1d06e94d46f885310208e04eb3af)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:54.640380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:54.706088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b85a44a714445685828bf50632cc51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:54.707430 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:54.714002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1162/2326] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:54 2022] DELETE /identity/v3/users/29b85a44a714445685828bf50632cc51 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:55.106517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:55.131761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4d1d06e94d46f885310208e04eb3af'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.143397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:55.147399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:55.170243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4d1d06e94d46f885310208e04eb3af'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.171765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4d1d06e94d46f885310208e04eb3af'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.193764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4d1d06e94d46f885310208e04eb3af'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.312167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6478350d-4109-40d1-806b-d270d3219b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4d1d06e94d46f885310208e04eb3af'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.321090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1165/2327] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:54 2022] DELETE /identity/v3/users/fd4d1d06e94d46f885310208e04eb3af => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:55.327037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:55.327364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:55.327646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:55.339528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5e7ec-6653-4520-bcdb-d1f6b612f7e9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:55.365296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:55.367703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:55.457779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:55.458474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:55.458757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:55.459030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] PATH_INFO: `/v3/projects/1df9b2b830bb4b46ad1c1b7b0a60af8a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:55.461242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df9b2b830bb4b46ad1c1b7b0a60af8a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:55.463173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:55.523344 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1166/2328] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:55.529855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:55.531685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:55.631689 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf666464-1069-4242-8ff9-9b84bc906fcc None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:55.652265 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df9b2b830bb4b46ad1c1b7b0a60af8a', 'request_id': 'req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.668171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df9b2b830bb4b46ad1c1b7b0a60af8a', 'request_id': 'req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.678553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df9b2b830bb4b46ad1c1b7b0a60af8a', 'request_id': 'req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.760100 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc710cc-4f62-42aa-acb3-82b7a0898acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:55.761391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc710cc-4f62-42aa-acb3-82b7a0898acf service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:55.762265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc710cc-4f62-42aa-acb3-82b7a0898acf service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:55.762714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc710cc-4f62-42aa-acb3-82b7a0898acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:55.809298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] The token cache is being invalidate because project 1df9b2b830bb4b46ad1c1b7b0a60af8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:55.809298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.809298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4fa53f5c-3dfd-4cd2-b839-feb2102037cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:55.809298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1163/2329] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:55 2022] DELETE /identity/v3/projects/1df9b2b830bb4b46ad1c1b7b0a60af8a => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:55.809298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:55.809802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:55.809802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:55.869765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:55.875629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:55.913077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc710cc-4f62-42aa-acb3-82b7a0898acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:55.917079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc710cc-4f62-42aa-acb3-82b7a0898acf service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:55.939947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1167/2330] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:55.989609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1164/2331] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:55.999688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.000287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.000756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.010248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:56.059901 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:56.061445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:56.077579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:56.094913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1168/2332] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:56.100166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d7034-9f4e-466b-bb56-7e58f5261d37 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:56.112814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:56.114050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.114545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.115032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.115937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:56.116469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:56.119032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:56.159616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:56.159862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1165/2333] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:34:56.180281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d5a3d-1570-49d8-8cdb-ff4160305840 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:56.189715 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:56.190528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.190961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.191417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.192519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:56.198822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:56.292211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1169/2334] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:56.296222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:56.322299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4842a8c-365e-4aa3-97b3-dd39390664c0 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:56.323407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:56.472927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:56.473729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.474010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.474279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.474916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:56.476845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:56.618953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1166/2335] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:56 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:34:56.625921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:56.645131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdab9ac-df8e-4105-be1b-f2ebb2fba639 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:56.660054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:56.660955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.661237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.661508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.662199 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:56.666111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:56.688576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1170/2336] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:56.693378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:56.717603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cd0f3-ba79-4486-8f7d-c746fdb55161 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:56.731450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129b1d8-c089-42f8-b244-f9a46e9ef82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:56.733040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b1d8-c089-42f8-b244-f9a46e9ef82c admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.733510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b1d8-c089-42f8-b244-f9a46e9ef82c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.733950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129b1d8-c089-42f8-b244-f9a46e9ef82c admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.734884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129b1d8-c089-42f8-b244-f9a46e9ef82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:56.736775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129b1d8-c089-42f8-b244-f9a46e9ef82c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:56.753183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1167/2337] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:56.757067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:56.770428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c376d5-0466-4411-bf63-1a822f3e9110 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:56.780974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:56.782109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.782577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.783006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] PATH_INFO: `/v3/projects/38e9f131db4f4134b0b712ce747eab71/users/9bb5a62bc8bf4f7c89816bb99ec6ab73/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.803083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e9f131db4f4134b0b712ce747eab71, user_id=9bb5a62bc8bf4f7c89816bb99ec6ab73, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:56.806365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:56.884515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1171/2338] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:56 2022] PUT /identity/v3/projects/38e9f131db4f4134b0b712ce747eab71/users/9bb5a62bc8bf4f7c89816bb99ec6ab73/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:56.900766 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:56.901067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:56.901339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:56.946522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] MFA Rules not processed for user `9bb5a62bc8bf4f7c89816bb99ec6ab73`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:56.948248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:57.025535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:57.028310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1168/2339] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:57.038911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:57.042813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:57.144358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b075d-809b-492b-b402-bb825a7e52c9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:57.151015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:57.151733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:57.152013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:57.152285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:57.163466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:57.165259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:57.181505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1172/2340] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:57.194386 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:57.197063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:57.239270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:57.324383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21239f2-7c9e-40b4-898e-24d94d2e6d3b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:57.329592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75c893e-9c42-488e-84ec-ee93086c9fe3 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:57.329993 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:57.330714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:57.331070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:57.331432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] PATH_INFO: `/v3/projects/3eb54fe905044c9e8d77a4e151b08dde` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:57.332686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb54fe905044c9e8d77a4e151b08dde)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:57.335370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:57.336785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:57.337670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:57.338151 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:57.338542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] PATH_INFO: `/v3/users/9bb5a62bc8bf4f7c89816bb99ec6ab73` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:57.339963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb5a62bc8bf4f7c89816bb99ec6ab73)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:57.342077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:57.486000 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb54fe905044c9e8d77a4e151b08dde', 'request_id': 'req-35c16d45-3bc2-4abc-a810-99aa56ab66f0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.495268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb54fe905044c9e8d77a4e151b08dde', 'request_id': 'req-35c16d45-3bc2-4abc-a810-99aa56ab66f0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.501257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb54fe905044c9e8d77a4e151b08dde', 'request_id': 'req-35c16d45-3bc2-4abc-a810-99aa56ab66f0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.602358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb5a62bc8bf4f7c89816bb99ec6ab73'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.606657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:57.609276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] The token cache is being invalidate because project 3eb54fe905044c9e8d77a4e151b08dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:57.610312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.611196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-35c16d45-3bc2-4abc-a810-99aa56ab66f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.614436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1169/2341] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:57 2022] DELETE /identity/v3/projects/3eb54fe905044c9e8d77a4e151b08dde => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:57.619006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea95366-2c82-4567-a27e-42406382f042 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:57.619553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea95366-2c82-4567-a27e-42406382f042 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:57.619997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea95366-2c82-4567-a27e-42406382f042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:57.634841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb5a62bc8bf4f7c89816bb99ec6ab73'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.641472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb5a62bc8bf4f7c89816bb99ec6ab73'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.653755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb5a62bc8bf4f7c89816bb99ec6ab73'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.661080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-aea95366-2c82-4567-a27e-42406382f042 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:57.662547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-aea95366-2c82-4567-a27e-42406382f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:57.714055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a61c375a-95c1-41a8-b406-d89e1d0e6336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb5a62bc8bf4f7c89816bb99ec6ab73'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:57.728843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1173/2342] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:34:57 2022] DELETE /identity/v3/users/9bb5a62bc8bf4f7c89816bb99ec6ab73 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:57.735304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:57.738366 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:57.739561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:57.811563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1170/2343] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:57.821822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:57.822105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:57.822378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:57.863387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:57.865008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:57.903061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:57.912549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:57.934616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:58.022833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1174/2344] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:58.029701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:58.031844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:58.089798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1171/2345] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:34:58.093892 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:58.112540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ef5aa-6133-4e16-b589-e7375b966a3f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:58.137459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:58.140046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:58.144800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:58.144800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:58.144800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:34:58.144800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:58.147843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:58.182900 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1172/2346] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 00:34:58.184322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f6dc2-36f5-4f92-be84-e6386a9981ed None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:58.188991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:58.197937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:58.199050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:58.200785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:58.201268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:58.215644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f957f34-ae26-4f1c-90a8-f35a582f2210 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:58.221996 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:58.224214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:58.230281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:58.236515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:58.236515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:58.236515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:58.236515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:58.236515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:58.250300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1175/2347] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:34:58.347695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1173/2348] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:34:58 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:58.348584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:58.365680 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc003258-4805-4ef3-8227-8f1e4d4cf674 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:58.469578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:58.470768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:58.472262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:58.472262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:58.474187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:58.479402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:58.758405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:58.779690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1176/2349] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:34:58 2022] POST /identity/v3/users => generated 287 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:34:58.807484 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:58.897240 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c0903-e79d-45bc-96fe-afc5b6b78b56 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:58.907624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:58.907624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:58.908117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:58.908117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:58.908779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:58.911504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:58.952178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1174/2350] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:58.956385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:58.966957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b0f707-f184-485b-850c-d48201cd0874 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:58.976018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb7329-dfa4-4932-9cb9-94b28bf38c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:58.977153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7329-dfa4-4932-9cb9-94b28bf38c35 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:58.977440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7329-dfa4-4932-9cb9-94b28bf38c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:58.977717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb7329-dfa4-4932-9cb9-94b28bf38c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:58.978374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb7329-dfa4-4932-9cb9-94b28bf38c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:58.980018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deb7329-dfa4-4932-9cb9-94b28bf38c35 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:58.998611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1177/2351] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:34:58.999432 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:59.010796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a094aa-37cf-49f4-9737-d3f0792197bc admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:59.045328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:59.047662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:59.048159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:59.048627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] PATH_INFO: `/v3/projects/3101620baddd42c48179a6f411bbd33e/users/0797d03452e9419d93e79ccf1c222391/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:59.074276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3101620baddd42c48179a6f411bbd33e, user_id=0797d03452e9419d93e79ccf1c222391, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:59.077396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:59.078605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f20e-6bb5-4a08-8c0c-27d36854eb79 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:59.078907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f20e-6bb5-4a08-8c0c-27d36854eb79 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:59.079191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9f20e-6bb5-4a08-8c0c-27d36854eb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:59.098214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b6a9f20e-6bb5-4a08-8c0c-27d36854eb79 None None] MFA Rules not processed for user `9f87028b4b624ebb829a4b61da5f9b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:59.099447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b6a9f20e-6bb5-4a08-8c0c-27d36854eb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:59.154218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1178/2352] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:59.185287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:34:59.321706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1175/2353] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:34:58 2022] PUT /identity/v3/projects/3101620baddd42c48179a6f411bbd33e/users/0797d03452e9419d93e79ccf1c222391/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:34:59.327463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:59.327788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:59.328113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:59.350517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:34:59.351908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:59.467820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1179/2354] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:34:59.476114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:59.478814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:34:59.486830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:34:59.511560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649ce0f-1186-48f0-95e3-1b2fbef0f5cd admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:59.528498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:59.529781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:59.530399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:59.540823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] PATH_INFO: `/v3/projects/38e9f131db4f4134b0b712ce747eab71` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:59.543326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e9f131db4f4134b0b712ce747eab71)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:59.549812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:59.710400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad0e0c-f2b4-42bd-9e1c-ec73d3a1efeb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:34:59.731352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4a0a5-2ab3-4601-9294-1b1f688d247a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:34:59.733034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a0a5-2ab3-4601-9294-1b1f688d247a service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:59.734288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a0a5-2ab3-4601-9294-1b1f688d247a service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:59.735825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4a0a5-2ab3-4601-9294-1b1f688d247a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:59.746273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e9f131db4f4134b0b712ce747eab71', 'request_id': 'req-c55a41f6-e8de-4a8a-8121-72ce1720c626'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:59.756007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a0a5-2ab3-4601-9294-1b1f688d247a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:34:59.762168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4a0a5-2ab3-4601-9294-1b1f688d247a service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:34:59.762603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e9f131db4f4134b0b712ce747eab71', 'request_id': 'req-c55a41f6-e8de-4a8a-8121-72ce1720c626'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:59.777580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e9f131db4f4134b0b712ce747eab71', 'request_id': 'req-c55a41f6-e8de-4a8a-8121-72ce1720c626'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:59.781525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1180/2355] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:34:59.900680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] The token cache is being invalidate because project 38e9f131db4f4134b0b712ce747eab71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:34:59.901577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:59.903641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c55a41f6-e8de-4a8a-8121-72ce1720c626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:34:59.906008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1176/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:34:59 2022] DELETE /identity/v3/projects/38e9f131db4f4134b0b712ce747eab71 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:34:59.910899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:34:59.911544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:34:59.912065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:34:59.933412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:00.027091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:00.027091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:00.038219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:00.195423 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1181/2357] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:35:00.203712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:00.204313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:00.204829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:00.265713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:00.267005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:00.291092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1177/2358] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:35:00.292661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:00.305762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793c7cf-64e8-4ab5-a963-9f4cd2cdb541 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:00.350597 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:00.351381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:00.356778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:00.356778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:00.356778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:35:00.356778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:00.356778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:00.377234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1182/2359] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:35:00.377727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:00.398546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f611dd-c234-4daa-a315-6d474a14cddb None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:00.415247 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:00.415957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:00.416367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:00.416704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:00.417458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:00.429703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:00.548427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1178/2360] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:35:00 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:00.549144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:00.560048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b567-04ea-4a62-9129-9309a6a18823 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:00.585755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:00.749478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:00.762955 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:00.763636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:00.763918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:00.764192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:00.764841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:00.782425 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:00.937696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1183/2361] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:35:00 2022] POST /identity/v3/users => generated 286 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:35:00.940398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:00.951271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4661-a26d-4fbe-948b-577344ad8e1b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:00.976415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:00.977009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:00.977445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:00.977723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:00.978412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:00.985082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:01.009872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1179/2362] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:01.013486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:01.025771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19582c7e-d9d4-4441-b2bc-91843ad7adfb admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:01.046924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3480662-8833-44c5-ab6e-c6440d384b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:01.047817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3480662-8833-44c5-ab6e-c6440d384b19 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:01.048280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3480662-8833-44c5-ab6e-c6440d384b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:01.050116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3480662-8833-44c5-ab6e-c6440d384b19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:01.051164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3480662-8833-44c5-ab6e-c6440d384b19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:01.059954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3480662-8833-44c5-ab6e-c6440d384b19 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:01.086724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1184/2363] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:01.087755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:01.102784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef77c3a-cedc-413d-9fac-2c7488c16f35 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:01.113451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:01.114186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:01.114467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:01.114745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] PATH_INFO: `/v3/projects/eeb0838e2fa84e51bd802278436c700b/users/f407a3f998b143098d844d3982f1fc67/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:01.165534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb0838e2fa84e51bd802278436c700b, user_id=f407a3f998b143098d844d3982f1fc67, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:01.168292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:01.314761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1180/2364] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:01 2022] PUT /identity/v3/projects/eeb0838e2fa84e51bd802278436c700b/users/f407a3f998b143098d844d3982f1fc67/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:01.325604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ec2ca-a253-43e4-b97e-f5aa62508947 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:01.325788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ec2ca-a253-43e4-b97e-f5aa62508947 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:01.326066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ec2ca-a253-43e4-b97e-f5aa62508947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:01.407882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ea0ec2ca-a253-43e4-b97e-f5aa62508947 None None] MFA Rules not processed for user `f407a3f998b143098d844d3982f1fc67`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:01.409650 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ea0ec2ca-a253-43e4-b97e-f5aa62508947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:01.521398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1185/2365] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:01.532619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:01.534662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:01.666490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df52de55-b210-4b63-9173-582dc826e259 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:01.681842 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:01.682649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:01.682941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:01.683218 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:01.708241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:01.719438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:01.747929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1181/2366] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:01.963287 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:02.063918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff650-c6a9-471a-a98c-313af5b69cb1 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:02.064399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff650-c6a9-471a-a98c-313af5b69cb1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:02.064501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ff650-c6a9-471a-a98c-313af5b69cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:02.127862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:02.133838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-283ff650-c6a9-471a-a98c-313af5b69cb1 None None] MFA Rules not processed for user `03110f36df2d444fad6ff04bc308be4d`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:02.135074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-283ff650-c6a9-471a-a98c-313af5b69cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:02.277354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1186/2367] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:02.290376 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:02.300373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81e613-befe-42dc-b7dd-eb27631aafa8 service glance] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:02.305312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:02.305312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:02.305312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:02.305749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:02.309977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:02.458856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:02.462171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:02.494843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1182/2368] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:03.001806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a9c63-b6c1-424b-a2cc-406e9a5e6210 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:03.002415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a9c63-b6c1-424b-a2cc-406e9a5e6210 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:03.002907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a9c63-b6c1-424b-a2cc-406e9a5e6210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:03.045832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a00a9c63-b6c1-424b-a2cc-406e9a5e6210 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:03.048059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a00a9c63-b6c1-424b-a2cc-406e9a5e6210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:03.086599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:03.170662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1187/2369] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:35:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:03.196257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:03.198676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:03.348550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:03.381605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472ac30-5dec-4389-bf16-4f581a08ad8d service glance] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:03.393866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:03.395195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:03.401253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:03.401550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:03.436400 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:03.438582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:03.450024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1183/2370] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:12.192442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:12.193034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:12.193034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:12.257593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] MFA Rules not processed for user `0797d03452e9419d93e79ccf1c222391`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:12.258855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:12.396560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1188/2371] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:12.401318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:12.403917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:12.507520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cd45b-dcb1-4c27-b48e-f786b8c4f718 service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:12.512860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:12.518119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:12.518119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:12.518119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:12.518119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:12.518119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:12.668218 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1184/2372] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:35:12 2022] POST /identity/v3/projects => generated 371 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:12.676056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:12.718728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e548a-197b-4c61-8ded-4bdb68592409 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:12.822097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:12.852915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:12.854011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:12.854375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:12.854987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:12.862237 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:12.865270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:12.987861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1189/2373] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:35:12 2022] POST /identity/v3/users => generated 292 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:12.991532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:13.013774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13124116-a513-4c7c-b6af-0c687091fdc4 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:13.020246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:13.025206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:13.025206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:13.025206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:13.025206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:13.025206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:13.038301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1185/2374] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:13.042242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:13.060338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c358765a-cc58-44fe-83c3-1d17413a8d97 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:13.076669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:13.077522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:13.077871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:13.078264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] PATH_INFO: `/v3/projects/eea2a896babe4ab2b7136af9bb4c3851/users/4f21395607594c5dad1c0d1b13ad17cf/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:13.097821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea2a896babe4ab2b7136af9bb4c3851, user_id=4f21395607594c5dad1c0d1b13ad17cf, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:13.100793 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:13.170845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1190/2375] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:13 2022] PUT /identity/v3/projects/eea2a896babe4ab2b7136af9bb4c3851/users/4f21395607594c5dad1c0d1b13ad17cf/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:35:13.175713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:13.193834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b50ac8-8fd1-4013-a59d-36a14a625513 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:13.263159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:13.306496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:13.307113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:13.308445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:13.308726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:13.309373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:13.310959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:13.329519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1186/2376] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:35:13.343665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:13.368404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50901b72-44b7-4647-a0ca-ef14b6272997 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:13.374268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a936743-12de-4f72-8ce3-b3bd1396c0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:13.375256 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a936743-12de-4f72-8ce3-b3bd1396c0db admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:13.375835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a936743-12de-4f72-8ce3-b3bd1396c0db admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:13.376323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a936743-12de-4f72-8ce3-b3bd1396c0db admin admin] PATH_INFO: `/v3/projects/eea2a896babe4ab2b7136af9bb4c3851/users/4f21395607594c5dad1c0d1b13ad17cf/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:13.378936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a936743-12de-4f72-8ce3-b3bd1396c0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea2a896babe4ab2b7136af9bb4c3851, user_id=4f21395607594c5dad1c0d1b13ad17cf, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:13.382024 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a936743-12de-4f72-8ce3-b3bd1396c0db admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:13.401968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1191/2377] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:13 2022] PUT /identity/v3/projects/eea2a896babe4ab2b7136af9bb4c3851/users/4f21395607594c5dad1c0d1b13ad17cf/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:35:15.912565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:15.921583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:15.937096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64c27c-ef9b-4928-b5cc-8198eac86ef6 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:16.022270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:16.023198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:16.027280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:16.027280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] PATH_INFO: `/v3/users/c2de6d1cc2e4439aa4973174c1e37b75` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:16.027280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2de6d1cc2e4439aa4973174c1e37b75)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:16.029039 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:16.308708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2de6d1cc2e4439aa4973174c1e37b75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:16.345940 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2de6d1cc2e4439aa4973174c1e37b75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:16.347872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2de6d1cc2e4439aa4973174c1e37b75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:16.367437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2de6d1cc2e4439aa4973174c1e37b75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:16.485037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2de6d1cc2e4439aa4973174c1e37b75'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:16.509723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1187/2378] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:35:15 2022] DELETE /identity/v3/users/c2de6d1cc2e4439aa4973174c1e37b75 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:35:16.515437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:16.515437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:16.515437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:16.581201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:16.582627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:16.751371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1192/2379] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:16.759117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:16.760663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:16.930003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:16.942874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a47a3-79ca-48d4-9faa-e71a5e851b00 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:16.950267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39629496-302f-4d41-a38f-49c78ba955e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:16.952463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39629496-302f-4d41-a38f-49c78ba955e6 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:16.952648 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39629496-302f-4d41-a38f-49c78ba955e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:16.952885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39629496-302f-4d41-a38f-49c78ba955e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:16.974903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39629496-302f-4d41-a38f-49c78ba955e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:16.976998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39629496-302f-4d41-a38f-49c78ba955e6 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:16.992263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1188/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:18.082949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:18.082949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:18.126814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c08156-c987-4a82-8e9e-eda58b3e3d4b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:18.136469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:18.137655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:18.138325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:18.138806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] PATH_INFO: `/v3/projects/2039a0e1619d46a8927b63797d34dc5e` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:18.141388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2039a0e1619d46a8927b63797d34dc5e)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:18.143668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:18.314912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2039a0e1619d46a8927b63797d34dc5e', 'request_id': 'req-788b27c6-1972-46e0-84fe-8b4f17800dcb'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:18.330900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2039a0e1619d46a8927b63797d34dc5e', 'request_id': 'req-788b27c6-1972-46e0-84fe-8b4f17800dcb'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:18.360956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2039a0e1619d46a8927b63797d34dc5e', 'request_id': 'req-788b27c6-1972-46e0-84fe-8b4f17800dcb'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:18.448361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] The token cache is being invalidate because project 2039a0e1619d46a8927b63797d34dc5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:35:18.449516 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:18.450352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-788b27c6-1972-46e0-84fe-8b4f17800dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:18.453022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1193/2381] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:35:18 2022] DELETE /identity/v3/projects/2039a0e1619d46a8927b63797d34dc5e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:18.822459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09711c-e2bc-48b5-aecd-047a2d5b3050 None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:18.823102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09711c-e2bc-48b5-aecd-047a2d5b3050 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:18.823102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09711c-e2bc-48b5-aecd-047a2d5b3050 None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:18.824285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1189/2382] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:35:18.834616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c5ea2-606d-4fbe-91d2-7d4b5e6647ef None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:18.838428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c5ea2-606d-4fbe-91d2-7d4b5e6647ef None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:18.838428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c5ea2-606d-4fbe-91d2-7d4b5e6647ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:18.904082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2d6c5ea2-606d-4fbe-91d2-7d4b5e6647ef None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:18.908661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2d6c5ea2-606d-4fbe-91d2-7d4b5e6647ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:18.930851 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:19.151844 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1194/2383] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:35:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:20.086699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09711c-e2bc-48b5-aecd-047a2d5b3050 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:20.089655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-de09711c-e2bc-48b5-aecd-047a2d5b3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:20.200066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09711c-e2bc-48b5-aecd-047a2d5b3050 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:20.207008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ad45c-6a03-41b4-9d36-7d17f481b49c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:20.207953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad45c-6a03-41b4-9d36-7d17f481b49c service glance] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:20.208405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad45c-6a03-41b4-9d36-7d17f481b49c service glance] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:20.208850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ad45c-6a03-41b4-9d36-7d17f481b49c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:20.222619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad45c-6a03-41b4-9d36-7d17f481b49c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:20.224555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ad45c-6a03-41b4-9d36-7d17f481b49c service glance] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:20.241532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1190/2384] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:20.541177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ab9cd-a634-4d0e-8ef2-f01d6e6af925 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:20.541177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ab9cd-a634-4d0e-8ef2-f01d6e6af925 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:20.542315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ab9cd-a634-4d0e-8ef2-f01d6e6af925 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:20.542995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1195/2385] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 00:35:20.565032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f67d32-a9aa-446d-b0e1-51a87d574dfc None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:20.565032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f67d32-a9aa-446d-b0e1-51a87d574dfc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:20.565032 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f67d32-a9aa-446d-b0e1-51a87d574dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:20.589070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-19f67d32-a9aa-446d-b0e1-51a87d574dfc None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:20.593362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-19f67d32-a9aa-446d-b0e1-51a87d574dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:20.756555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1191/2386] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:35:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:20.771743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ab9cd-a634-4d0e-8ef2-f01d6e6af925 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:20.774158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4e9ab9cd-a634-4d0e-8ef2-f01d6e6af925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:20.803621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ab9cd-a634-4d0e-8ef2-f01d6e6af925 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:20.822684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd6171-7571-418e-9e5d-ff39f5ddb2ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:20.824181 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd6171-7571-418e-9e5d-ff39f5ddb2ea service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:20.824642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd6171-7571-418e-9e5d-ff39f5ddb2ea service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:20.825101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd6171-7571-418e-9e5d-ff39f5ddb2ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:20.837835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd6171-7571-418e-9e5d-ff39f5ddb2ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:20.839923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd6171-7571-418e-9e5d-ff39f5ddb2ea service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:20.857806 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1196/2387] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:20.983036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba497b8b-6546-4cf8-8deb-4b7e4c1259db None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:20.983770 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba497b8b-6546-4cf8-8deb-4b7e4c1259db None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:20.984229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba497b8b-6546-4cf8-8deb-4b7e4c1259db None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:20.985848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1192/2388] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:35:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:35:21.001419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c200f-e126-4d3c-b326-e824b71bd8b7 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:21.002166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c200f-e126-4d3c-b326-e824b71bd8b7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:21.002638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c200f-e126-4d3c-b326-e824b71bd8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:21.075799 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-353c200f-e126-4d3c-b326-e824b71bd8b7 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:21.078419 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-353c200f-e126-4d3c-b326-e824b71bd8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:21.099991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1197/2389] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:35:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:21.114977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba497b8b-6546-4cf8-8deb-4b7e4c1259db None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:21.132192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba497b8b-6546-4cf8-8deb-4b7e4c1259db None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:21.137397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:21.138457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:21.139026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:21.140270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:21.156577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:21.158525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:21.180700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1193/2390] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:23.430260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c03b-95c7-445b-8484-8526bedb06b9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:23.430771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c03b-95c7-445b-8484-8526bedb06b9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:23.430876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0c03b-95c7-445b-8484-8526bedb06b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:23.479513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-07f0c03b-95c7-445b-8484-8526bedb06b9 None None] MFA Rules not processed for user `4f21395607594c5dad1c0d1b13ad17cf`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:23.481348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-07f0c03b-95c7-445b-8484-8526bedb06b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:23.589854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1198/2391] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:23.606040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:23.607504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:23.632585 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc31a4-9c17-4eb3-92af-f0a35513e1ad service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:23.638757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:23.639824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:23.640100 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:23.640529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:23.642553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:23.760420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:23.762473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c989160-a6db-4ca2-a8dc-242419df2770 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:23.780169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1194/2392] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:24.344611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:24.344961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:24.345301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:24.366965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] MFA Rules not processed for user `41fecf50555a4effb8aca217296bbad6`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:24.368288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:24.429724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1199/2393] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:26.551283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41710-c6fe-45ec-89f4-030658b3db64 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:26.552238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41710-c6fe-45ec-89f4-030658b3db64 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:26.552522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d41710-c6fe-45ec-89f4-030658b3db64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:26.581852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-16d41710-c6fe-45ec-89f4-030658b3db64 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:26.581852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-16d41710-c6fe-45ec-89f4-030658b3db64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:26.607593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:26.717507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1195/2394] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:26.729646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:26.732473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:26.757869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:26.855309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53e130-3a79-4522-a809-1fa56ea34805 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:26.865854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20446615-6547-432f-8c43-cef9be402c07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:26.866716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446615-6547-432f-8c43-cef9be402c07 service swift] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:26.867134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446615-6547-432f-8c43-cef9be402c07 service swift] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:26.867611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446615-6547-432f-8c43-cef9be402c07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:26.882079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20446615-6547-432f-8c43-cef9be402c07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:26.898757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20446615-6547-432f-8c43-cef9be402c07 service swift] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:26.912606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1200/2395] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:26.955540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:26.955540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:26.955540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:26.993366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:26.995690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:27.012502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b4298-ff80-444f-9bdf-ff23fc0a4848 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:27.012786 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b4298-ff80-444f-9bdf-ff23fc0a4848 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:27.013191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b4298-ff80-444f-9bdf-ff23fc0a4848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:27.025996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1196/2396] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:35:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:27.042332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c50b4298-ff80-444f-9bdf-ff23fc0a4848 None None] MFA Rules not processed for user `0797d03452e9419d93e79ccf1c222391`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:27.042650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:27.044350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c50b4298-ff80-444f-9bdf-ff23fc0a4848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:27.055287 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1201/2397] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:35:27.057670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753e519-34d3-4c7a-8e61-4339764dec6c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:27.063670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:27.064365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:27.064710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:27.065205 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:27.086424 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:27.088694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:27.102673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1197/2398] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:28.822107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4b99-7503-4465-8c2f-a473ec4d595d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:28.823169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4b99-7503-4465-8c2f-a473ec4d595d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:28.823630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece4b99-7503-4465-8c2f-a473ec4d595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:28.862731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3ece4b99-7503-4465-8c2f-a473ec4d595d None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:28.864497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3ece4b99-7503-4465-8c2f-a473ec4d595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:28.875128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1202/2399] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 00:35:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:28.914186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:28.935466 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306b124-6be2-410b-91d6-da08fea76c6e service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:28.943257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:28.945018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:28.945303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:28.945580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:28.957710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:28.960398 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:28.973505 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1198/2400] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:30.402111 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc5f9a-7d5a-4c9f-be9f-032e4e67c7cf None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:30.403077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc5f9a-7d5a-4c9f-be9f-032e4e67c7cf None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:30.403442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc5f9a-7d5a-4c9f-be9f-032e4e67c7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:30.423871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9edc5f9a-7d5a-4c9f-be9f-032e4e67c7cf None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:30.427079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9edc5f9a-7d5a-4c9f-be9f-032e4e67c7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:30.438868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1203/2401] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:35:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:30.455488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:30.468369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c7d17-d7a0-4ffa-a891-324f58605296 service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:30.481898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:30.484732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:30.484732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:30.484732 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:30.493626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:30.495543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:30.504947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1199/2402] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 00:35:31.488684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225a8d-4987-429d-be32-63994e181898 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:31.489209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225a8d-4987-429d-be32-63994e181898 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:31.489209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225a8d-4987-429d-be32-63994e181898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:31.507928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9b225a8d-4987-429d-be32-63994e181898 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:31.509041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9b225a8d-4987-429d-be32-63994e181898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:31.516879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1204/2403] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 00:35:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:31.528643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:31.539831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404e8f0-4a95-4373-8918-a72ba22cb165 service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:31.545426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbbd91c-6f57-431c-b383-f7761ea373f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:31.546161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbd91c-6f57-431c-b383-f7761ea373f6 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:31.547320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbd91c-6f57-431c-b383-f7761ea373f6 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:31.547617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbbd91c-6f57-431c-b383-f7761ea373f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:31.556827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbd91c-6f57-431c-b383-f7761ea373f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:31.558416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbbd91c-6f57-431c-b383-f7761ea373f6 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:31.577269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1200/2404] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:34.651789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225a8d-4987-429d-be32-63994e181898 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:34.654145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9b225a8d-4987-429d-be32-63994e181898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:34.754917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225a8d-4987-429d-be32-63994e181898 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:34.760661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:34.761403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:34.761792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:34.762505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] PATH_INFO: `/v3/users/9f87028b4b624ebb829a4b61da5f9b9d` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:34.763961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f87028b4b624ebb829a4b61da5f9b9d)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:34.767321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:35.180966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f87028b4b624ebb829a4b61da5f9b9d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:35.195205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f87028b4b624ebb829a4b61da5f9b9d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:35.197487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f87028b4b624ebb829a4b61da5f9b9d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:35.204546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f87028b4b624ebb829a4b61da5f9b9d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:35.273114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f87028b4b624ebb829a4b61da5f9b9d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:35.276065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:35.280933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1205/2405] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:35:34 2022] DELETE /identity/v3/users/9f87028b4b624ebb829a4b61da5f9b9d => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:35.285610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:35.286271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:35.286271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:35.306666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:35.307923 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:35.386385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1201/2406] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:35.401773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:35.404035 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:35.489307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc54ccf1-468a-4932-8bbb-5021862aa3db admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:35.493192 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07efc9e-5bf8-4ca7-a7b0-a98cd21c7eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:35.509415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07efc9e-5bf8-4ca7-a7b0-a98cd21c7eff service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:35.509415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07efc9e-5bf8-4ca7-a7b0-a98cd21c7eff service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:35.509415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07efc9e-5bf8-4ca7-a7b0-a98cd21c7eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:35.509415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07efc9e-5bf8-4ca7-a7b0-a98cd21c7eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:35.509415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07efc9e-5bf8-4ca7-a7b0-a98cd21c7eff service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:35.515887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1206/2407] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:35.863079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:35.866099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:35.884147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ed957-e24b-4916-979b-556d5cac77b8 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:35.893827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:35.894844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:35.895129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:35.895430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] PATH_INFO: `/v3/projects/0fb873acb9dc45d98bb0ccde8ee50a5a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:35.897179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb873acb9dc45d98bb0ccde8ee50a5a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:35.905696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.026515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb873acb9dc45d98bb0ccde8ee50a5a', 'request_id': 'req-5bb0f039-78f9-4956-83bb-565e95ed8ad7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:36.032340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb873acb9dc45d98bb0ccde8ee50a5a', 'request_id': 'req-5bb0f039-78f9-4956-83bb-565e95ed8ad7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:36.039054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb873acb9dc45d98bb0ccde8ee50a5a', 'request_id': 'req-5bb0f039-78f9-4956-83bb-565e95ed8ad7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:36.045708 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:36.137805 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] The token cache is being invalidate because project 0fb873acb9dc45d98bb0ccde8ee50a5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:35:36.138102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:36.138856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-5bb0f039-78f9-4956-83bb-565e95ed8ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:36.141829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1202/2408] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:35:35 2022] DELETE /identity/v3/projects/0fb873acb9dc45d98bb0ccde8ee50a5a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:36.145884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.146401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.146518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.166710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:36.168069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:36.229783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1207/2409] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:36.243695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.243992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.244294 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.266354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:36.267650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:36.275726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1203/2410] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:36.280505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.290485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f797c-08a0-4f4d-8643-9ef7713b8152 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.295728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.296267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.296545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.296820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.297462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:35:36.297763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.299483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.312646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1208/2411] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:35:36.317895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.329793 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c057f9-6133-47ea-a2f6-c0c21ae26a07 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.334909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.335555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.335836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.336150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.336791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.338841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.406210 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.408633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:36.460435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1204/2412] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:35:36 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:36.465383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.492773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658698-7f18-4db3-a12a-bd0933b97db1 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.526137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:36.540561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4676117-0970-4e6a-b1cc-af93ec8aa709 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.546180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.551693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.551693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.551693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.551693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.551693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.564744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.565353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.565633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.565966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.568018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:36.580083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.581664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.591832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1209/2413] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:35:36.632591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1205/2414] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:35:36 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:36.639638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.652736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee44881-b7e8-4ec6-aee5-0f19bbe98bfc service placement] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.658004 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.658620 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.658882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.659154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.659831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.661408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.675123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1210/2415] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:36.676995 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.687078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7abce7-c80a-41c1-bca7-63702c58a032 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.692709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c3d39-27b0-4e80-a5f7-b9fcb3ed2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.695374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c3d39-27b0-4e80-a5f7-b9fcb3ed2147 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.695374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c3d39-27b0-4e80-a5f7-b9fcb3ed2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.695374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c3d39-27b0-4e80-a5f7-b9fcb3ed2147 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.695374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c3d39-27b0-4e80-a5f7-b9fcb3ed2147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.696869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c3d39-27b0-4e80-a5f7-b9fcb3ed2147 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.718138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1206/2416] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:36.728249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.739143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901fd932-d158-42fc-9ae2-7431523fd463 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:36.743905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:36.744561 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.744931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.745186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] PATH_INFO: `/v3/projects/a22da44baa6b472e91d99fbcb03b583c/users/322afd3893ac4c2ba4ab4d449f4d9159/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.759805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22da44baa6b472e91d99fbcb03b583c, user_id=322afd3893ac4c2ba4ab4d449f4d9159, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:36.762515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:36.864837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1211/2417] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:36 2022] PUT /identity/v3/projects/a22da44baa6b472e91d99fbcb03b583c/users/322afd3893ac4c2ba4ab4d449f4d9159/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:36.870176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bb836-67e1-4efa-99bf-45e5c91c88d9 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:36.870546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bb836-67e1-4efa-99bf-45e5c91c88d9 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:36.871664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bb836-67e1-4efa-99bf-45e5c91c88d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:36.893935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0f2bb836-67e1-4efa-99bf-45e5c91c88d9 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:36.898659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0f2bb836-67e1-4efa-99bf-45e5c91c88d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:36.960310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1207/2418] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:36.974074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:36.977896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:37.035007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aca4b6-7296-4855-8c61-b34d5a0e5b10 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:37.038720 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:37.039349 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:37.039695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:37.039976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:37.049927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:37.052176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:37.062960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1212/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:39.423340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:39.423340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:39.423340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:39.445633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:39.447033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:39.504332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:39.520801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1208/2420] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:35:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:35:39.536512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:39.539350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:39.689375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe22d7-697a-45de-8a8a-653365119610 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:39.705244 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a8691-936e-416f-babc-88ca26c417ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:39.712690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8691-936e-416f-babc-88ca26c417ba service swift] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:39.712690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8691-936e-416f-babc-88ca26c417ba service swift] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:39.712690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a8691-936e-416f-babc-88ca26c417ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:39.724213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a8691-936e-416f-babc-88ca26c417ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:39.728778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8a8691-936e-416f-babc-88ca26c417ba service swift] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:39.759110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1213/2421] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:39.996435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:39.998106 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:40.027987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf012e95-2c83-41b5-b58d-c3d0cdd9af28 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:40.035374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:40.038104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:40.038104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:40.038104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] PATH_INFO: `/v3/users/f407a3f998b143098d844d3982f1fc67` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:40.038548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f407a3f998b143098d844d3982f1fc67)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:40.048960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:40.777506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f407a3f998b143098d844d3982f1fc67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:40.837375 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f407a3f998b143098d844d3982f1fc67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:40.841974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f407a3f998b143098d844d3982f1fc67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:40.870499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f407a3f998b143098d844d3982f1fc67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:40.971794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f407a3f998b143098d844d3982f1fc67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:40.978931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1209/2422] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:35:39 2022] DELETE /identity/v3/users/f407a3f998b143098d844d3982f1fc67 => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:40.983767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:40.984456 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:40.984595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:41.019575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:41.021098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:41.045721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:41.231636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1214/2423] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:41.278384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:41.281968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:41.423329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0858-7409-4350-aa16-625aa037f78d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:41.434254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb0b04-1b77-4415-bb4d-c859d6ec8308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:41.434879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb0b04-1b77-4415-bb4d-c859d6ec8308 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:41.435160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb0b04-1b77-4415-bb4d-c859d6ec8308 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:41.443661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb0b04-1b77-4415-bb4d-c859d6ec8308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:41.460506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb0b04-1b77-4415-bb4d-c859d6ec8308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:41.462161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb0b04-1b77-4415-bb4d-c859d6ec8308 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:41.507932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1210/2424] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:42.440647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:42.440647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:42.451392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03a3bc0-9142-497e-9b0d-f6d763908318 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:42.515415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:42.516122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:42.516398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:42.520582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] PATH_INFO: `/v3/projects/eeb0838e2fa84e51bd802278436c700b` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:42.524055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb0838e2fa84e51bd802278436c700b)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:42.527368 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:42.994521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb0838e2fa84e51bd802278436c700b', 'request_id': 'req-4112bf98-620c-41ab-b59a-806fe8a1bb1a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:43.002709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb0838e2fa84e51bd802278436c700b', 'request_id': 'req-4112bf98-620c-41ab-b59a-806fe8a1bb1a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:43.021398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb0838e2fa84e51bd802278436c700b', 'request_id': 'req-4112bf98-620c-41ab-b59a-806fe8a1bb1a'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:43.113062 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] The token cache is being invalidate because project eeb0838e2fa84e51bd802278436c700b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:35:43.115081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:43.115081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-4112bf98-620c-41ab-b59a-806fe8a1bb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:35:43.116508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1215/2425] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:35:42 2022] DELETE /identity/v3/projects/eeb0838e2fa84e51bd802278436c700b => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:43.121309 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.121851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.121851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.165966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:43.172414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:43.308920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1211/2426] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:43.318426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.319061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.319401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.342488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:43.344661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:43.355919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1216/2427] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:43.360657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:43.370631 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112ae30-c8a7-4095-9a5f-d545857bf5a6 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:43.376131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:43.376443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.376888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.377014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.377641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:35:43.378060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:43.380010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:43.381954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:43.400709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:43.406823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1212/2428] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:35:43.418679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543bba7-c58a-4439-a903-96dd89abe181 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:43.435146 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:43.439495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.439495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.439495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.439495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:43.439857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:43.570870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1217/2429] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:35:43 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:35:43.575193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:43.603009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d790f86-1b23-458b-a6d5-31e5a3647644 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:43.721690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:43.721690 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.722783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.723047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.723812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:43.727520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:43.855574 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1213/2430] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:35:43 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:43.856623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:43.875838 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f40b32-21be-497a-aef3-53d79be996f8 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:43.887964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:43.888856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.889378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.889930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.891438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:43.893452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:43.928193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1218/2431] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:43.943497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:43.962743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979959d-372d-47d9-a08a-70c39abce028 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:43.975344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32c8c3-86d2-4aed-93d8-8db5e8f83bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:43.976175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32c8c3-86d2-4aed-93d8-8db5e8f83bab admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:43.976711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32c8c3-86d2-4aed-93d8-8db5e8f83bab admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:43.977087 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32c8c3-86d2-4aed-93d8-8db5e8f83bab admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:43.978581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32c8c3-86d2-4aed-93d8-8db5e8f83bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:43.981130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32c8c3-86d2-4aed-93d8-8db5e8f83bab admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:44.007583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1214/2432] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:44.010125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:44.041285 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d57a6-b79b-4f0e-9ec9-0c1206fe1e5b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:44.047283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:44.048412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:44.048992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:44.049418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] PATH_INFO: `/v3/projects/b8f2a58683be4aa486cebdf7aa8e0d83/users/f950ecd21b5a4612bff847604ad8efdd/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:44.078371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f2a58683be4aa486cebdf7aa8e0d83, user_id=f950ecd21b5a4612bff847604ad8efdd, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:44.084531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:44.170283 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1219/2433] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:44 2022] PUT /identity/v3/projects/b8f2a58683be4aa486cebdf7aa8e0d83/users/f950ecd21b5a4612bff847604ad8efdd/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:35:44.176695 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf532b2-616c-4aa3-963c-6d3061a1aabf None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:44.177238 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf532b2-616c-4aa3-963c-6d3061a1aabf None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:44.177500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf532b2-616c-4aa3-963c-6d3061a1aabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:44.205292 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1cf532b2-616c-4aa3-963c-6d3061a1aabf None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:44.206869 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1cf532b2-616c-4aa3-963c-6d3061a1aabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:44.350236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1215/2434] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:44.365136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:44.366987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:44.419519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:44.434820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10d76b-9c22-4563-a713-8f7039eea09f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:44.440227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12989da6-3f33-42e7-bf94-b26b01ecf5a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:44.441208 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12989da6-3f33-42e7-bf94-b26b01ecf5a4 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:44.441360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12989da6-3f33-42e7-bf94-b26b01ecf5a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:44.441643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12989da6-3f33-42e7-bf94-b26b01ecf5a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:44.455761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12989da6-3f33-42e7-bf94-b26b01ecf5a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:44.457375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12989da6-3f33-42e7-bf94-b26b01ecf5a4 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:44.473319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1220/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:35:47.433385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:47.434440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:47.434769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:47.480840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:47.481569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:47.481569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:47.509176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] MFA Rules not processed for user `f950ecd21b5a4612bff847604ad8efdd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:47.514431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:47.553617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] MFA Rules not processed for user `322afd3893ac4c2ba4ab4d449f4d9159`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:47.554880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:47.703772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1216/2436] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:35:47.749033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1221/2437] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:35:47.754332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:47.756271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:47.767929 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:47.781864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f18a9-5279-4b1c-9f15-3187553bb46c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:47.782559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:47.783712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:47.822587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:47.823263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:47.823547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:47.823825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:47.827984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:47.832747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:47.857966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3dafd-6182-43a7-ad11-0b1ce04fa93e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:47.862770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:47.863481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:47.863762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:47.864036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:47.875725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:47.877361 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:47.899269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1222/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:35:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:35:48.017173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1217/2439] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:35:47 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:48.020992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:48.038379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ba3af-e3da-492e-8e77-dad6d44240f2 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:48.175855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:48.177087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:48.177087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:48.177839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:48.178494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:48.180921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:48.305394 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1223/2440] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:35:48 2022] POST /identity/v3/users => generated 295 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:35:48.307843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:48.341513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d1b78-f346-432e-9a84-bd0abdb01492 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:48.364349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:48.365140 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:48.365522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:48.365895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:48.369166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:48.372910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:48.388004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1218/2441] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:48.392546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:48.408927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:48.416630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7d421-5943-4b8e-9ae0-2678759130ab admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:48.432959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:48.434307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:48.434999 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:48.435460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] PATH_INFO: `/v3/projects/ad2ab7aaac7341739eee2f43ea9f161f/users/a818a59ba73c455c9d11092655901cd9/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:48.454268 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2ab7aaac7341739eee2f43ea9f161f, user_id=a818a59ba73c455c9d11092655901cd9, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:48.457842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:48.558881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1224/2442] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:48 2022] PUT /identity/v3/projects/ad2ab7aaac7341739eee2f43ea9f161f/users/a818a59ba73c455c9d11092655901cd9/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:48.567120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:48.582840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945dc8b4-6c7b-4ba6-a221-184c6ca8ccdc admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:48.744615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7b4ad-66f9-4a90-83a9-a7b4f90ab339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:48.745616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7b4ad-66f9-4a90-83a9-a7b4f90ab339 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:48.746123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7b4ad-66f9-4a90-83a9-a7b4f90ab339 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:48.746661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7b4ad-66f9-4a90-83a9-a7b4f90ab339 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:48.747877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7b4ad-66f9-4a90-83a9-a7b4f90ab339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:48.752531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7b4ad-66f9-4a90-83a9-a7b4f90ab339 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:48.768757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1219/2443] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:35:48.773567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:48.805177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236adfa-8781-4588-85b5-928116aee2f5 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:48.823056 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50551beb-7e70-4499-ba9e-766f949b5642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:48.823867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50551beb-7e70-4499-ba9e-766f949b5642 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:48.824058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50551beb-7e70-4499-ba9e-766f949b5642 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:48.824324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50551beb-7e70-4499-ba9e-766f949b5642 admin admin] PATH_INFO: `/v3/projects/ad2ab7aaac7341739eee2f43ea9f161f/users/a818a59ba73c455c9d11092655901cd9/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:48.826292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50551beb-7e70-4499-ba9e-766f949b5642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2ab7aaac7341739eee2f43ea9f161f, user_id=a818a59ba73c455c9d11092655901cd9, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:48.830527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50551beb-7e70-4499-ba9e-766f949b5642 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:48.886871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1225/2444] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:35:48 2022] PUT /identity/v3/projects/ad2ab7aaac7341739eee2f43ea9f161f/users/a818a59ba73c455c9d11092655901cd9/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:35:50.312913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:50.320525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:50.320525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:50.354502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] MFA Rules not processed for user `f950ecd21b5a4612bff847604ad8efdd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:50.358986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:50.503930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1220/2445] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:53.516116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a7608-7ea8-44c6-8541-f0f72eddbde1 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:53.516924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a7608-7ea8-44c6-8541-f0f72eddbde1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:53.517281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a7608-7ea8-44c6-8541-f0f72eddbde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:53.542609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-8a7a7608-7ea8-44c6-8541-f0f72eddbde1 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:53.544357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8a7a7608-7ea8-44c6-8541-f0f72eddbde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:53.659328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1226/2446] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 00:35:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:35:53.688974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:35:53.691169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:53.803427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78add393-c5fa-4440-b8d3-543dac183880 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:35:53.818675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb82b39-b54d-47ad-b7eb-5f59a91ecf1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:35:53.819335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82b39-b54d-47ad-b7eb-5f59a91ecf1b service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:53.819821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82b39-b54d-47ad-b7eb-5f59a91ecf1b service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:53.820275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb82b39-b54d-47ad-b7eb-5f59a91ecf1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:53.837500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82b39-b54d-47ad-b7eb-5f59a91ecf1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:35:53.845104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb82b39-b54d-47ad-b7eb-5f59a91ecf1b service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:35:53.851403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:35:53.857570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1221/2447] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:35:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:35:54.400931 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:35:54.401216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:35:54.401638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:35:54.457007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] MFA Rules not processed for user `7b317f585f8a4da3abe24ab5cf4ee0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:35:54.461687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:35:54.605307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1227/2448] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:35:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:36:00.081984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:36:00.082449 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:36:00.082553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:36:00.124896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] MFA Rules not processed for user `a818a59ba73c455c9d11092655901cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:36:00.126190 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:00.208471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1222/2449] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:36:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:36:00.223096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:36:00.224607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:00.331511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91332e04-db7d-4f97-b50e-5b8ad45530fa None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:36:00.340707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:36:00.341640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:36:00.347948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:36:00.348270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:36:00.370916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:36:00.437526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:36:00.439688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:36:00.451330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1228/2450] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:36:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:36:10.204807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=34 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:36:11.008800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=21 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:36:15.378250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:36:15.379754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:15.438776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb0e911-a693-4147-b6a0-cb69a444226c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:36:15.442384 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:36:15.442978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:36:15.443286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:36:15.443558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:36:15.445289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:15.521326 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:36:15.522827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cbe0a-510f-487a-8d23-78f3a4098a18 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:36:15.534067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1223/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:36:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:36:17.697116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:36:17.707883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19052-8a74-4bc4-8e29-712940d21a0b service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:36:17.713525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:36:17.714406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:36:17.714899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:36:17.715434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:36:17.718013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:17.813822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:36:17.815322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45827d6-e26c-431e-bcb8-61d09a814da9 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:36:17.829905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1229/2452] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:36:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:36:25.528702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=50 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:36:27.824828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=30 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:36:30.009179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:36:30.009685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:36:30.009796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:36:30.031022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] MFA Rules not processed for user `322afd3893ac4c2ba4ab4d449f4d9159`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:36:30.032370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:30.039142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1224/2453] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:36:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:36:31.118432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede64dc-4d54-4514-9fb1-420ced2e1914 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:36:31.119370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede64dc-4d54-4514-9fb1-420ced2e1914 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:36:31.119821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede64dc-4d54-4514-9fb1-420ced2e1914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:36:31.147756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1ede64dc-4d54-4514-9fb1-420ced2e1914 None None] MFA Rules not processed for user `c0aa06eea99f4011ab85ec51b0e60f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:36:31.149064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1ede64dc-4d54-4514-9fb1-420ced2e1914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:36:31.226168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1230/2454] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:36:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:36:40.024831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:36:41.211188 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=25 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:03.097304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:03.107162 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b72dd-00bd-4c21-9083-85295894486f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:03.111075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:03.111745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:03.112026 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:03.112355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:03.121602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:03.123048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:03.132955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1225/2455] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:10.733020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:10.733486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:10.733603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:10.753280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] MFA Rules not processed for user `fcc8eeb749a5457f98492da99199528c`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:37:10.754435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:10.810452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1231/2456] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:37:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:37:13.128685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:18.580088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:18.589896 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccfc292-fe02-48b7-a89b-a5826bc6d9ef service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:18.594217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:18.594361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:18.594698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:18.595035 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:18.597066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:18.642953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:18.644617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7481efc9-dceb-4b44-b160-9a85d44ec0e8 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:18.656783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1226/2457] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:20.803692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=25 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:22.520605 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:22.530128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07303a48-9f5c-4414-9208-35e3d13485f9 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:22.533745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:22.534336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:22.534612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:22.534885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:22.537290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:22.550392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:22.551909 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:22.562573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1232/2458] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:22.765308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893355c-3d68-4155-88c2-da50f7b8e651 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:22.765813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893355c-3d68-4155-88c2-da50f7b8e651 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:22.765906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893355c-3d68-4155-88c2-da50f7b8e651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:22.789092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a893355c-3d68-4155-88c2-da50f7b8e651 None None] MFA Rules not processed for user `f950ecd21b5a4612bff847604ad8efdd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:37:22.790256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a893355c-3d68-4155-88c2-da50f7b8e651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:22.797211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1227/2459] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:37:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:37:32.557890 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=6 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:32.780504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=9 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:35.608230 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:35.609628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:35.667475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188a2a-988f-4674-8757-3f4af85bb8a9 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:35.671054 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:35.671682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:35.671952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:35.672207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:35.673928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:35.685704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:35.687118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:35.696011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1233/2460] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:35 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:45.691754 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=26 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:47.929079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:47.929617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:47.929617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:47.948426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] MFA Rules not processed for user `7b317f585f8a4da3abe24ab5cf4ee0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:37:47.949669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:47.956512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1228/2461] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:37:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:37:48.509847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:48.519016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0a05e-31a0-4981-a260-d653ef816419 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:48.522722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:48.523367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:48.523644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:48.523912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:48.525613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:48.534910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:48.536345 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:48.544258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1234/2462] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:48.983013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:48.984649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:49.055416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7aa3b-8c74-47be-ac54-fbd0c5f544ce None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:49.060477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:49.061078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:49.061351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:49.061617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:49.065059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:49.075042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:49.076649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:49.087719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1229/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:58.540856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=6 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:59.082300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=35 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:37:59.356673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:59.359259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:59.376884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e531fa-a550-4a9d-9f2b-66c9c45a3c3f service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:59.382623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:59.383296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:59.383543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:59.384000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:59.385719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:59.398474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:59.401167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:59.411729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1235/2464] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:37:59.677954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:37:59.690506 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b74ddb-d765-44fb-bb6f-73857c9a4469 service placement] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:37:59.694405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:37:59.695018 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:37:59.695323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:37:59.695594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:37:59.697866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:37:59.710683 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:37:59.712290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:37:59.723007 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1230/2465] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:37:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:00.550245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:00.562107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc520-7bcf-4ad3-a238-195d15896751 service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:00.567830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:00.568426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:00.568902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:00.569176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:00.570883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:00.584246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:00.585832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:00.597465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1236/2466] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:09.723208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:10.592751 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=12 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:10.942121 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:10.952776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b81e9-d395-4822-9eec-f04505e7e429 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:10.956772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:10.957367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:10.957647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:10.957921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:10.959922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:11.026947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:11.028504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:11.039175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1231/2467] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:14.808405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:14.809870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:14.824974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfe7e2-4710-4242-8773-218928b38266 service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:14.833546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:14.834191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:14.834472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:14.834744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:14.836541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:14.849204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:14.850812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:14.861393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1237/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:14.912663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:14.913179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:14.926154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e412cc-339c-4229-bc2c-20949145edee service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:14.930789 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:14.931445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:14.931729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:14.932005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] PATH_INFO: `/v3/users/c0aa06eea99f4011ab85ec51b0e60f2c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:14.933173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0aa06eea99f4011ab85ec51b0e60f2c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:14.935175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:15.127919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0aa06eea99f4011ab85ec51b0e60f2c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.139605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0aa06eea99f4011ab85ec51b0e60f2c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.140285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0aa06eea99f4011ab85ec51b0e60f2c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.147110 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0aa06eea99f4011ab85ec51b0e60f2c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.204914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-52ae701e-fe16-4991-a9ef-feb19ffe10c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0aa06eea99f4011ab85ec51b0e60f2c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.211275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1232/2469] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:14 2022] DELETE /identity/v3/users/c0aa06eea99f4011ab85ec51b0e60f2c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:15.454678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:15.465542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b26a86-46c3-49f8-b9fb-4ad787e03f9e service placement] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:15.524335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:15.524961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:15.525242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:15.525513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] PATH_INFO: `/v3/projects/9bd235f1f9f44df4b2e3dbd0c05b3a83` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:15.526631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd235f1f9f44df4b2e3dbd0c05b3a83)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:15.528606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:15.628674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd235f1f9f44df4b2e3dbd0c05b3a83', 'request_id': 'req-f4a86f22-10d4-4e0c-af26-b82c4d310735'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.634301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd235f1f9f44df4b2e3dbd0c05b3a83', 'request_id': 'req-f4a86f22-10d4-4e0c-af26-b82c4d310735'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.641009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd235f1f9f44df4b2e3dbd0c05b3a83', 'request_id': 'req-f4a86f22-10d4-4e0c-af26-b82c4d310735'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.722018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] The token cache is being invalidate because project 9bd235f1f9f44df4b2e3dbd0c05b3a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:15.722722 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.722995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f4a86f22-10d4-4e0c-af26-b82c4d310735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:15.724484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1238/2470] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:15 2022] DELETE /identity/v3/projects/9bd235f1f9f44df4b2e3dbd0c05b3a83 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:15.729606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:15.729989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:15.730388 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:15.750841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:15.752089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:15.813613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1233/2471] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:15.821642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:15.822141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:15.822255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:15.843371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:15.844564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:15.852357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1239/2472] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:15.856652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:15.867255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eebcf-629b-446b-9769-d0217605aa69 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:15.872064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:15.872693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:15.873617 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:15.873897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:15.874657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:38:15.874955 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:15.876674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:15.878665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:15.890465 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1234/2473] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:38:15.894821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:15.904385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca8a13-7b5d-4d35-b332-4cd0c3813086 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:15.909109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:15.909696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:15.909971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:15.910236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:15.910846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:15.912771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.014147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1240/2474] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:15 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:38:16.018403 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.028936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d96a63-6e1d-4b36-8dd1-0654142d303d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.085781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.086382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.086671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.086931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.087580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.089485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.178979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1235/2475] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:16 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:16.183154 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.194219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0617acb-afd1-456d-8f82-8ac43249ae3b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.198878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.201264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.201264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.201264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.201264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.202676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.213145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1241/2476] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:16.222290 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.233267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebac99-7bdd-46b7-a97f-f50517959d3a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.238002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283aa32-ee95-41fa-83aa-1db3c2f181b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.238595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283aa32-ee95-41fa-83aa-1db3c2f181b7 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.238873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283aa32-ee95-41fa-83aa-1db3c2f181b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.239146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283aa32-ee95-41fa-83aa-1db3c2f181b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.239815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283aa32-ee95-41fa-83aa-1db3c2f181b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.241392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283aa32-ee95-41fa-83aa-1db3c2f181b7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.252047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1236/2477] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:16.256262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.266324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb229-18de-4699-9fd9-38fa097cd88f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.270748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.271441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.271717 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.271987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] PATH_INFO: `/v3/projects/3ed022c402ac447097679a96a7a0df79/users/3b2986294ae945cfb3ac03ed71192ae2/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.286655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed022c402ac447097679a96a7a0df79, user_id=3b2986294ae945cfb3ac03ed71192ae2, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.289305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.367772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1242/2478] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:16 2022] PUT /identity/v3/projects/3ed022c402ac447097679a96a7a0df79/users/3b2986294ae945cfb3ac03ed71192ae2/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:38:16.379346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.379656 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.379916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.410659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] MFA Rules not processed for user `3b2986294ae945cfb3ac03ed71192ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:16.411883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:16.474494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1237/2479] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:16.479008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.493859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:16.556937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb287f4c-f109-4ada-bc3a-a759eccebcf5 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.561610 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.562209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.562489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.562763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.563422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.565319 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.676426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1243/2480] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:16 2022] POST /identity/v3/projects => generated 373 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:38:16.680699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.692369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b883a-46a8-43a5-b557-0959c8e301d5 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.746741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.747362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.747647 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.748407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.749027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.750926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.759609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:16.864497 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1238/2481] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:16 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:16.868480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.879474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62a076-8e22-4c30-b9dd-94098a71b8d9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.884079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.884673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.884950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.885809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.885904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.887475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:16.899536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1244/2482] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:16.903725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:16.913724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d5ae5-bb1e-4091-bd0f-7982a009ca35 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:16.918335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:16.918974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:16.919281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:16.919557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] PATH_INFO: `/v3/projects/7b4f0cbd07844dcd8dfa50ca2897dec6/users/149af21d0e0d4964829c06a5f1d31b6f/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:16.934172 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4f0cbd07844dcd8dfa50ca2897dec6, user_id=149af21d0e0d4964829c06a5f1d31b6f, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:16.937643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:17.001350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1239/2483] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:16 2022] PUT /identity/v3/projects/7b4f0cbd07844dcd8dfa50ca2897dec6/users/149af21d0e0d4964829c06a5f1d31b6f/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:17.005249 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:17.015745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943ded8-dd2f-49cc-9174-242d3c470123 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:17.069922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf95544-495e-44cf-8308-deaf1a96f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:17.070515 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95544-495e-44cf-8308-deaf1a96f218 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:17.070791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95544-495e-44cf-8308-deaf1a96f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:17.071061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf95544-495e-44cf-8308-deaf1a96f218 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:17.071729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf95544-495e-44cf-8308-deaf1a96f218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:17.073308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf95544-495e-44cf-8308-deaf1a96f218 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:17.082677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1245/2484] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:17.087074 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:17.097354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf8a73-49a2-4cb3-97d4-f3d2fb9b47e7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:17.102006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:17.102643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:17.102922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:17.103195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] PATH_INFO: `/v3/projects/7b4f0cbd07844dcd8dfa50ca2897dec6/users/149af21d0e0d4964829c06a5f1d31b6f/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:17.105133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4f0cbd07844dcd8dfa50ca2897dec6, user_id=149af21d0e0d4964829c06a5f1d31b6f, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:17.108423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:17.118463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1240/2485] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:17 2022] PUT /identity/v3/projects/7b4f0cbd07844dcd8dfa50ca2897dec6/users/149af21d0e0d4964829c06a5f1d31b6f/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:17.129373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:17.129685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:17.129954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:17.156382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] MFA Rules not processed for user `149af21d0e0d4964829c06a5f1d31b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:17.157573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:17.223540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1246/2486] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:17.240552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:17.242379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:17.266875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:17.268278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:17.281194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:17.304058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4713f6-1bc5-4849-8a91-1b6f1ba020e2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:17.307914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:17.308507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:17.308780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:17.309047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:17.318728 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:17.320303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:17.330033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1241/2487] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:17.348171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd89de1-ff8e-4510-8b26-573796518403 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:17.354280 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:17.354882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:17.355158 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:17.355466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:17.357180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:17.422679 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:17.424478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f29ddd-fe7e-427a-9dcb-1959e78119c7 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:17.434343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1247/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:38:18.272422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:18.363457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53763a0-af19-46e9-99d7-5964f07526ad service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:18.368875 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:18.369519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:18.369933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:18.370215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] PATH_INFO: `/v3/users/3b2986294ae945cfb3ac03ed71192ae2` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:18.371489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2986294ae945cfb3ac03ed71192ae2)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:18.376217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:18.587619 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2986294ae945cfb3ac03ed71192ae2'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:18.599342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:18.605717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2986294ae945cfb3ac03ed71192ae2'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:18.606919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2986294ae945cfb3ac03ed71192ae2'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:18.617601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2986294ae945cfb3ac03ed71192ae2'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:18.706606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2986294ae945cfb3ac03ed71192ae2'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:18.714203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1242/2489] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:18 2022] DELETE /identity/v3/users/3b2986294ae945cfb3ac03ed71192ae2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:18.719971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dada83-6162-4eea-b169-638fe5de649a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:18.720487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dada83-6162-4eea-b169-638fe5de649a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:18.720588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dada83-6162-4eea-b169-638fe5de649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:18.742987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-38dada83-6162-4eea-b169-638fe5de649a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:18.744300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-38dada83-6162-4eea-b169-638fe5de649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:18.817077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1248/2490] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:18.853090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:18.854999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:18.924055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6716298-483e-4783-8346-a3cd321c8b5a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:18.931553 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:18.932233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:18.932512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:18.932785 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:18.943841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:18.945286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:18.957013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1243/2491] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:19.483081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dada83-6162-4eea-b169-638fe5de649a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:19.483081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-38dada83-6162-4eea-b169-638fe5de649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:19.496711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dada83-6162-4eea-b169-638fe5de649a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:19.506498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:19.511875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:19.511875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:19.511875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] PATH_INFO: `/v3/projects/3ed022c402ac447097679a96a7a0df79` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:19.511875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed022c402ac447097679a96a7a0df79)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:19.511875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:19.658438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:19.672698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed022c402ac447097679a96a7a0df79', 'request_id': 'req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:19.681317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed022c402ac447097679a96a7a0df79', 'request_id': 'req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:19.688615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed022c402ac447097679a96a7a0df79', 'request_id': 'req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:19.762347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] The token cache is being invalidate because project 3ed022c402ac447097679a96a7a0df79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:19.765292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:19.765292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:19.768076 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1249/2492] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:19 2022] DELETE /identity/v3/projects/3ed022c402ac447097679a96a7a0df79 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:19.772340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:19.786139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59087e-8498-4964-a105-d0c85a08059f service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:19.863548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:19.864271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:19.865067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:19.865067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] PATH_INFO: `/v3/users/149af21d0e0d4964829c06a5f1d31b6f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:19.866020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149af21d0e0d4964829c06a5f1d31b6f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:19.868068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:20.135123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149af21d0e0d4964829c06a5f1d31b6f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:20.154748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149af21d0e0d4964829c06a5f1d31b6f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:20.156337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149af21d0e0d4964829c06a5f1d31b6f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:20.164561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149af21d0e0d4964829c06a5f1d31b6f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:20.263083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-d919d10c-0318-4959-b7ff-068bcabbfd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149af21d0e0d4964829c06a5f1d31b6f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:20.266182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:20.271489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1244/2493] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:19 2022] DELETE /identity/v3/users/149af21d0e0d4964829c06a5f1d31b6f => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:20.842327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:20.844042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:20.859526 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04169b-085b-4ad0-ada8-0bf1ca0f1bf8 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:20.917445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:20.918074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:20.918529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:20.918815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] PATH_INFO: `/v3/projects/7b4f0cbd07844dcd8dfa50ca2897dec6` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:20.919973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4f0cbd07844dcd8dfa50ca2897dec6)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:20.922646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.059437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4f0cbd07844dcd8dfa50ca2897dec6', 'request_id': 'req-beb44b57-2d58-4a9f-b28b-720f8d8d2266'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:21.066032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4f0cbd07844dcd8dfa50ca2897dec6', 'request_id': 'req-beb44b57-2d58-4a9f-b28b-720f8d8d2266'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:21.072569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4f0cbd07844dcd8dfa50ca2897dec6', 'request_id': 'req-beb44b57-2d58-4a9f-b28b-720f8d8d2266'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:21.208396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] The token cache is being invalidate because project 7b4f0cbd07844dcd8dfa50ca2897dec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:21.209626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:21.210300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-beb44b57-2d58-4a9f-b28b-720f8d8d2266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:21.211918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1250/2494] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:20 2022] DELETE /identity/v3/projects/7b4f0cbd07844dcd8dfa50ca2897dec6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:21.218011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.219012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.219129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.239918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:21.248642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:21.301187 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1245/2495] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:21.309295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.309698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.309929 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.332272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:21.333499 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:21.341965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1251/2496] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:21.347097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:21.357003 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee49b2a-ef6a-45b0-9f9b-fc45a2d34c0d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:21.362480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:21.363081 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.363393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.363666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.364318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:38:21.364618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:21.366215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.378359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1246/2497] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:38:21.382661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:21.392471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0bfbf-16e0-4397-9dde-734ae246b3cd None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:21.397271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:21.397862 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.398139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.398408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.399028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:21.401064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.511662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1252/2498] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:21 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:21.516533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:21.526733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3cffc-b8e8-408c-a936-9cfb320611fe admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:21.584650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:21.585462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.585571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.585817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.586474 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:21.588425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.688839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1247/2499] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:21 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:38:21.692975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:21.703842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d8d1e-711d-4583-b342-004cfb3c9023 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:21.709309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:21.709921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.710203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.710478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.711124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:21.712897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.726938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1253/2500] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:21.728340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:21.744288 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b422c3-cedb-462f-b584-bf3f0453e8e7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:21.745858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592ff08-a0db-4fb8-bf84-aa53ee7b45ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:21.746645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592ff08-a0db-4fb8-bf84-aa53ee7b45ed admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.747115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592ff08-a0db-4fb8-bf84-aa53ee7b45ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.747530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592ff08-a0db-4fb8-bf84-aa53ee7b45ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.748312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592ff08-a0db-4fb8-bf84-aa53ee7b45ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:21.750796 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592ff08-a0db-4fb8-bf84-aa53ee7b45ed admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.764555 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1248/2501] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:21.769725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:21.780559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b8744-7ebf-42e1-9ab9-f05452eebbe0 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:21.785502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:21.786152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.786428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.786697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] PATH_INFO: `/v3/projects/fd6e69d02f634f5a9a79642e6eb75bb5/users/11eeb695e5a242ccb54e0ec0b7168580/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.801934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6e69d02f634f5a9a79642e6eb75bb5, user_id=11eeb695e5a242ccb54e0ec0b7168580, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:21.804682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:21.807712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:21.942920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1254/2502] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:21 2022] PUT /identity/v3/projects/fd6e69d02f634f5a9a79642e6eb75bb5/users/11eeb695e5a242ccb54e0ec0b7168580/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:21.954566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:21.954851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:21.955132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:21.982963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] MFA Rules not processed for user `11eeb695e5a242ccb54e0ec0b7168580`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:21.984889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:22.040435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:22.050749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1249/2503] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:22.055346 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.117355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106b252-5961-42e6-b6e2-ae271e125c0d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.122121 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.125261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.125261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.125261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.125261 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.126238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.245323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1255/2504] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:22 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:38:22.246513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.258038 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49ec0c-b9ed-48f8-9314-79331511dd30 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.323816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.324073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.324364 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.324787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.325256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.327165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.434126 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1250/2505] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:22 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:22.438664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.449041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3559b-1c7a-43ea-909a-16ada3028890 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.453700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.454327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.454606 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.454876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.455550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.457275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.468260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1256/2506] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:22.472634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.482638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05981d-4ff2-4587-849b-bf548ddaf957 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.487848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.488486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.488764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.489036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] PATH_INFO: `/v3/projects/bdc42b5d97d54aabb18fa9310070a2b9/users/913733913ffe4e60b7b1352ec5d99954/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.504295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc42b5d97d54aabb18fa9310070a2b9, user_id=913733913ffe4e60b7b1352ec5d99954, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.506967 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.605874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1251/2507] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:22 2022] PUT /identity/v3/projects/bdc42b5d97d54aabb18fa9310070a2b9/users/913733913ffe4e60b7b1352ec5d99954/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:22.612117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.622170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dd04a-8908-4123-9504-1209a1cacd87 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.680875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf444692-e03e-4be3-8144-53eeb44b21df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.681490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444692-e03e-4be3-8144-53eeb44b21df admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.681768 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444692-e03e-4be3-8144-53eeb44b21df admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.682058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf444692-e03e-4be3-8144-53eeb44b21df admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.682682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444692-e03e-4be3-8144-53eeb44b21df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.684341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf444692-e03e-4be3-8144-53eeb44b21df admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.694133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1257/2508] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:22.698435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.709428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b276ff-85f0-4df3-8d09-019166d7dfba admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.714036 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.714679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.714959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.715250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] PATH_INFO: `/v3/projects/bdc42b5d97d54aabb18fa9310070a2b9/users/913733913ffe4e60b7b1352ec5d99954/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.717239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc42b5d97d54aabb18fa9310070a2b9, user_id=913733913ffe4e60b7b1352ec5d99954, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.720372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.730696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1252/2509] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:22 2022] PUT /identity/v3/projects/bdc42b5d97d54aabb18fa9310070a2b9/users/913733913ffe4e60b7b1352ec5d99954/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:22.742022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.742330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.742607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.770102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] MFA Rules not processed for user `913733913ffe4e60b7b1352ec5d99954`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:22.771356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:22.824995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:22.844191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1258/2510] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:22.856013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:22.857855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:22.917610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17d250-2bd0-4fb0-b26e-72aeee8c6dd3 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:22.922934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66c709-c76e-4f8d-8f01-0af701b542a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:22.923586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66c709-c76e-4f8d-8f01-0af701b542a3 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:22.924001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66c709-c76e-4f8d-8f01-0af701b542a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:22.924195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66c709-c76e-4f8d-8f01-0af701b542a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:22.934183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66c709-c76e-4f8d-8f01-0af701b542a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:22.935746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66c709-c76e-4f8d-8f01-0af701b542a3 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:22.946049 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1253/2511] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:22.976370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:23.040102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794926c5-b918-4df0-bc7e-8c9448aa3c5e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:23.044790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:23.045414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:23.045697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:23.045973 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] PATH_INFO: `/v3/users/11eeb695e5a242ccb54e0ec0b7168580` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:23.047449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11eeb695e5a242ccb54e0ec0b7168580)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:23.049374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:23.248765 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11eeb695e5a242ccb54e0ec0b7168580'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:23.263046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11eeb695e5a242ccb54e0ec0b7168580'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:23.267713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11eeb695e5a242ccb54e0ec0b7168580'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:23.271034 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11eeb695e5a242ccb54e0ec0b7168580'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:23.393842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11eeb695e5a242ccb54e0ec0b7168580'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:23.400811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1259/2512] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:22 2022] DELETE /identity/v3/users/11eeb695e5a242ccb54e0ec0b7168580 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:23.405267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:23.406316 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:23.406680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:23.425801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:23.427045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:23.475439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:23.490566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1254/2513] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:23.502755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:23.502755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:23.558512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcc841-f074-4693-9c93-89ff5b764d18 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:23.562752 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:23.563374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:23.563658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:23.563933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:23.572801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:23.574220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:23.588873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1260/2514] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:24.047767 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:24.049425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:24.062431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57d88-b5a9-488c-b5c6-a9ecd315dab6 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:24.074117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:24.074857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:24.075149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:24.075459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] PATH_INFO: `/v3/projects/fd6e69d02f634f5a9a79642e6eb75bb5` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:24.076653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6e69d02f634f5a9a79642e6eb75bb5)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:24.079178 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:24.204502 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6e69d02f634f5a9a79642e6eb75bb5', 'request_id': 'req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.210569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6e69d02f634f5a9a79642e6eb75bb5', 'request_id': 'req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.216868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6e69d02f634f5a9a79642e6eb75bb5', 'request_id': 'req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.298248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] The token cache is being invalidate because project fd6e69d02f634f5a9a79642e6eb75bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:24.298473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.299088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.300585 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1255/2515] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:24 2022] DELETE /identity/v3/projects/fd6e69d02f634f5a9a79642e6eb75bb5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:24.304663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:24.314709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be524db-f9c5-43f8-917a-7bac111e2df1 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:24.323148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:24.369821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:24.370578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:24.377291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:24.377291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] PATH_INFO: `/v3/users/913733913ffe4e60b7b1352ec5d99954` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:24.377291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] RBAC: Authorizing `identity:delete_user(user_id=913733913ffe4e60b7b1352ec5d99954)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:24.377291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:24.585627 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '913733913ffe4e60b7b1352ec5d99954'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.597347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '913733913ffe4e60b7b1352ec5d99954'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.598021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '913733913ffe4e60b7b1352ec5d99954'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.604788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '913733913ffe4e60b7b1352ec5d99954'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.681355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-e8c3cfc5-08fb-4578-ac57-6c784b8e006d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '913733913ffe4e60b7b1352ec5d99954'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:24.688472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1261/2516] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:24 2022] DELETE /identity/v3/users/913733913ffe4e60b7b1352ec5d99954 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:25.232866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:25.232866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:25.246279 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5e376-8c46-4eaf-b276-3ec7f8251f68 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:25.315878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:25.316801 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.317267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.317544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] PATH_INFO: `/v3/projects/bdc42b5d97d54aabb18fa9310070a2b9` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.320874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc42b5d97d54aabb18fa9310070a2b9)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:25.320874 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:25.425654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc42b5d97d54aabb18fa9310070a2b9', 'request_id': 'req-1a22a045-a4fb-4244-9d2f-9c80685af93a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:25.431359 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc42b5d97d54aabb18fa9310070a2b9', 'request_id': 'req-1a22a045-a4fb-4244-9d2f-9c80685af93a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:25.437616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc42b5d97d54aabb18fa9310070a2b9', 'request_id': 'req-1a22a045-a4fb-4244-9d2f-9c80685af93a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:25.521829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] The token cache is being invalidate because project bdc42b5d97d54aabb18fa9310070a2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:25.522118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:25.522760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1a22a045-a4fb-4244-9d2f-9c80685af93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:25.524393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1256/2517] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:25 2022] DELETE /identity/v3/projects/bdc42b5d97d54aabb18fa9310070a2b9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:25.529350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.529852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.529962 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.551150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:25.552367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:25.610446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1262/2518] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:25.618025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.618307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.618581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.626090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:25.638751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:25.640014 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:25.648395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1257/2519] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:25.652678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:25.663094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871aa2b-c104-433f-ae82-1640374c53f5 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:25.667695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:25.668288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.668564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.668837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.669484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:38:25.669784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:25.671406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:25.683149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1263/2520] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:38:25.687592 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:25.697006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a57a9-3b95-4f44-b486-6e4b5dbd6981 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:25.702393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:25.702982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.703286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.703562 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.704214 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:25.706741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:25.785974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1258/2521] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:25 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:25.790136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:25.800690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f2cc5-aea9-468f-88fd-7551088a12a4 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:25.817064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:25.857462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:25.858072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.858850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.858850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.859282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:25.861363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:25.948400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1264/2522] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:25 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:25.952469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:25.965002 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf67d2-2f83-45be-849b-62b7e2fc7ac6 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:25.969622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:25.981581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:25.981581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:25.981581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:25.981581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:25.981581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:25.983457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1259/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:25.987482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:25.997576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9afc5-5bd7-4604-b23f-156bee11b2e6 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.002618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d712f-e5ec-41bb-95cf-8c5e72d84360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.003311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d712f-e5ec-41bb-95cf-8c5e72d84360 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.003585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d712f-e5ec-41bb-95cf-8c5e72d84360 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.003852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d712f-e5ec-41bb-95cf-8c5e72d84360 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.004507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d712f-e5ec-41bb-95cf-8c5e72d84360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.006049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d712f-e5ec-41bb-95cf-8c5e72d84360 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.016021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1265/2524] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:26.020778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.030626 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e54b90-c1bf-4118-acd2-4abcb548866d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.035343 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.035980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.036257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.036529 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] PATH_INFO: `/v3/projects/5de17245ca0543258f177046415b89da/users/ef79c5b2987f49e6b5f63a8d21e72bd8/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.050771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de17245ca0543258f177046415b89da, user_id=ef79c5b2987f49e6b5f63a8d21e72bd8, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.053430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.144421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1260/2525] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:26 2022] PUT /identity/v3/projects/5de17245ca0543258f177046415b89da/users/ef79c5b2987f49e6b5f63a8d21e72bd8/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:26.155782 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.156067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.156340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.183898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] MFA Rules not processed for user `ef79c5b2987f49e6b5f63a8d21e72bd8`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:26.185096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:26.249415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1266/2526] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:26.254361 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.319538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e65770-3000-42ed-bf80-5a9cd99ef94e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.324699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.325519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.325870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.326220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.326914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.329136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.430596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1261/2527] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:26 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:26.434733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.447816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48197-875a-426a-9427-e6350e49b4eb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.521064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.521685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.521963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.522242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.522865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.524827 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.619243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1267/2528] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:26 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:26.628755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.638972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6833e7-4393-468f-8b1b-9c0133468615 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.643722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.644312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.644586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.644857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.645490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.647114 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.661984 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1262/2529] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:38:26.666139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.677084 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ba20a-4369-45eb-98fb-770d5acd610a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.681738 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.682386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.682666 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.682937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] PATH_INFO: `/v3/projects/2c6d3b78c92c4deb95766969099dc0c5/users/a7b10406497d4aaeb4d6299546220b37/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.698461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6d3b78c92c4deb95766969099dc0c5, user_id=a7b10406497d4aaeb4d6299546220b37, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.701153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.703329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:26.784630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1268/2530] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:26 2022] PUT /identity/v3/projects/2c6d3b78c92c4deb95766969099dc0c5/users/a7b10406497d4aaeb4d6299546220b37/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:26.788663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.799440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103beee0-5da8-4aef-8e01-254b2af83395 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.855048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9a5663-d690-44a5-86fc-0f9501445873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.855691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5663-d690-44a5-86fc-0f9501445873 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.855966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5663-d690-44a5-86fc-0f9501445873 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.856234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5663-d690-44a5-86fc-0f9501445873 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.856861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a5663-d690-44a5-86fc-0f9501445873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.858434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9a5663-d690-44a5-86fc-0f9501445873 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.868895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1263/2531] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:26.873204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:26.883527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51a961-5e07-44f9-a922-4c353358dced admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:26.888744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:26.889391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.889664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.889934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] PATH_INFO: `/v3/projects/2c6d3b78c92c4deb95766969099dc0c5/users/a7b10406497d4aaeb4d6299546220b37/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.891906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6d3b78c92c4deb95766969099dc0c5, user_id=a7b10406497d4aaeb4d6299546220b37, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:26.894548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:26.904953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1269/2532] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:26 2022] PUT /identity/v3/projects/2c6d3b78c92c4deb95766969099dc0c5/users/a7b10406497d4aaeb4d6299546220b37/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:26.916119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:26.916404 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:26.916681 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:26.936523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:26.944851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] MFA Rules not processed for user `a7b10406497d4aaeb4d6299546220b37`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:26.946042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:27.013632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1264/2533] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:38:27.024956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:27.026701 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:27.083446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30b45a-0b18-4716-a365-758b73a32dfe admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:27.087331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff25b6-806d-4a61-a2d4-84bf838c264a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:27.087924 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff25b6-806d-4a61-a2d4-84bf838c264a service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:27.088202 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff25b6-806d-4a61-a2d4-84bf838c264a service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:27.088472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff25b6-806d-4a61-a2d4-84bf838c264a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:27.097922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff25b6-806d-4a61-a2d4-84bf838c264a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:27.099384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff25b6-806d-4a61-a2d4-84bf838c264a service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:27.108535 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1270/2534] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:27.172405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:27.234248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ef4c9-051d-4cdb-b53f-f6e6c7122e0e None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:27.238927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:27.239598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:27.239902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:27.240171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] PATH_INFO: `/v3/users/ef79c5b2987f49e6b5f63a8d21e72bd8` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:27.241306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef79c5b2987f49e6b5f63a8d21e72bd8)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:27.243538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:27.453813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef79c5b2987f49e6b5f63a8d21e72bd8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:27.467042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef79c5b2987f49e6b5f63a8d21e72bd8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:27.467786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef79c5b2987f49e6b5f63a8d21e72bd8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:27.475442 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef79c5b2987f49e6b5f63a8d21e72bd8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:27.570883 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef79c5b2987f49e6b5f63a8d21e72bd8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:27.579063 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1265/2535] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:27 2022] DELETE /identity/v3/users/ef79c5b2987f49e6b5f63a8d21e72bd8 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:27.584714 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:27.590983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:27.590983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:27.609584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:27.610788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:27.691101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1271/2536] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:27.703979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:27.705515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:27.721985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:27.766560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341d374-f6cc-4df9-ad66-4a5e45db23e9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:27.770131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:27.771387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:27.771387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:27.771387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:27.780654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:27.791109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:27.792788 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1266/2537] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:28.327415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:28.328920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:28.349840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7a64b-a145-46a8-a062-a308909ae45c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:28.354790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:28.355519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:28.355760 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:28.356006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] PATH_INFO: `/v3/projects/5de17245ca0543258f177046415b89da` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:28.357238 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de17245ca0543258f177046415b89da)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:28.359152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:28.467504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de17245ca0543258f177046415b89da', 'request_id': 'req-8f715557-0d3e-4fee-a05b-4569ee920abf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.473223 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de17245ca0543258f177046415b89da', 'request_id': 'req-8f715557-0d3e-4fee-a05b-4569ee920abf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.479619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de17245ca0543258f177046415b89da', 'request_id': 'req-8f715557-0d3e-4fee-a05b-4569ee920abf'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.551119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] The token cache is being invalidate because project 5de17245ca0543258f177046415b89da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:28.551119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.551119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.552480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1272/2538] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:28 2022] DELETE /identity/v3/projects/5de17245ca0543258f177046415b89da => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:28.556524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:28.567022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ef7f9-c0ec-4709-807f-66b258d86fb9 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:28.633293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:28.633964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:28.634241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:28.634517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] PATH_INFO: `/v3/users/a7b10406497d4aaeb4d6299546220b37` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:28.635701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b10406497d4aaeb4d6299546220b37)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:28.637706 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:28.840513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b10406497d4aaeb4d6299546220b37'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.874079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b10406497d4aaeb4d6299546220b37'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.875362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b10406497d4aaeb4d6299546220b37'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.883119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b10406497d4aaeb4d6299546220b37'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.950224 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b53bbf3b-513c-424c-95fc-43f80ba21399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b10406497d4aaeb4d6299546220b37'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:28.957806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1267/2539] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:28 2022] DELETE /identity/v3/users/a7b10406497d4aaeb4d6299546220b37 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:38:29.364731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:29.365224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:29.379444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715557-0d3e-4fee-a05b-4569ee920abf admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:29.384830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:29.437367 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:29.437991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:29.438317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:29.438590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] PATH_INFO: `/v3/projects/2c6d3b78c92c4deb95766969099dc0c5` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:29.439906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6d3b78c92c4deb95766969099dc0c5)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:29.441880 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:29.545175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6d3b78c92c4deb95766969099dc0c5', 'request_id': 'req-91fc1d4b-77d8-434d-a47c-c67820377c31'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:29.550739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6d3b78c92c4deb95766969099dc0c5', 'request_id': 'req-91fc1d4b-77d8-434d-a47c-c67820377c31'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:29.556855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6d3b78c92c4deb95766969099dc0c5', 'request_id': 'req-91fc1d4b-77d8-434d-a47c-c67820377c31'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:29.633209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] The token cache is being invalidate because project 2c6d3b78c92c4deb95766969099dc0c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:29.633578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:29.634250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-91fc1d4b-77d8-434d-a47c-c67820377c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:29.635936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1273/2540] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:29 2022] DELETE /identity/v3/projects/2c6d3b78c92c4deb95766969099dc0c5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:38:29.641215 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:29.641654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:29.641765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:29.663408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:29.664723 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:29.696362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:29.726199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1268/2541] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:29.734077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:29.734369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:29.734704 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:29.754933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:29.756175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:29.764203 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1274/2542] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:29.769135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:29.779011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ccdbbe-541b-412b-b6f6-5e317b920bf7 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:29.783832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:29.784426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:29.784717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:29.784983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:29.785622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:38:29.785927 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:29.787537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:29.799418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1269/2543] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:38:29.803833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:29.813143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617cfdd-87bd-4fc3-a745-71b27d6af074 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:29.818439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3c821-d8c9-451e-b123-ee4c35571aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:29.819025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3c821-d8c9-451e-b123-ee4c35571aaa admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:29.819331 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3c821-d8c9-451e-b123-ee4c35571aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:29.822520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3c821-d8c9-451e-b123-ee4c35571aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:29.822520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3c821-d8c9-451e-b123-ee4c35571aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:29.822520 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3c821-d8c9-451e-b123-ee4c35571aaa admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:29.956495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1275/2544] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:29 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:29.957646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:29.968590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c5252-9939-4583-91e3-192d0db2c9b5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.041746 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.042472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.042813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.043150 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.043905 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.045963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.133080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1270/2545] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:29 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:30.133080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.134098 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f8be2-08af-490e-93cf-ccddf0f2b3f7 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.134444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f8be2-08af-490e-93cf-ccddf0f2b3f7 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.134812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f8be2-08af-490e-93cf-ccddf0f2b3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.143772 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c16bb-7a09-4310-8502-973d6322d7ed admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.148604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.149235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.149514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.149791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.150436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.152336 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.155965 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c50f8be2-08af-490e-93cf-ccddf0f2b3f7 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:30.157186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c50f8be2-08af-490e-93cf-ccddf0f2b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:30.163391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1271/2546] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:30.174590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.184680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597f0b1-5948-4925-8a25-ee0a4961b1be admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.190129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.190906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.191410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.191517 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.192127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.193705 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.203894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1272/2547] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:38:30.208084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.214070 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1276/2548] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 00:38:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:30.218334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9242931-205b-4a33-8921-7db394f51f0c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.224344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.224991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.225270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.225541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] PATH_INFO: `/v3/projects/2a905baf0b9c4c148f14213b0132d6fe/users/31b60743a2a246dab586f3a0688f949f/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.244847 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a905baf0b9c4c148f14213b0132d6fe, user_id=31b60743a2a246dab586f3a0688f949f, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.247649 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.337415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1273/2549] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:30 2022] PUT /identity/v3/projects/2a905baf0b9c4c148f14213b0132d6fe/users/31b60743a2a246dab586f3a0688f949f/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:38:30.349113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.349403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.349682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.365774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.367536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:30.376157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:30.381185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] MFA Rules not processed for user `31b60743a2a246dab586f3a0688f949f`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:30.382428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:30.445199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4310b-4a0e-4367-88b0-83a0d4ee8bd0 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.450457 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.450893 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1277/2550] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:30.451057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.451363 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.451635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.456278 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.457217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:30.511134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.512823 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.521298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1274/2551] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:38:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 00:38:30.521810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787693f-750c-4a5b-ba7b-29a6e7dcb9e4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.527317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.528013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.528359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.528696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.529403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.531832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.687130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1278/2552] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:30 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:30.691367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.706651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafe609-8d47-4962-9f78-933c8d65ec81 service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.762762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.763397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.763680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.763953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.764577 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.766638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.887054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1275/2553] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:30 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:38:30.890971 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.902732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b7288-1c45-4f3f-809b-290ac4c31c52 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.907132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.908363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.908667 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.908936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.909576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.911320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:30.922196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1279/2554] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:30.926260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:30.936714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4429f-0eca-4ab8-a39f-0f932af3f178 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:30.941803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:30.942518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:30.942798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:30.943073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] PATH_INFO: `/v3/projects/61b2499065aa4c05b660c1098f1fe618/users/c7dbf4f50dae470cbf2bd8f73bab4ed9/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:30.962167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b2499065aa4c05b660c1098f1fe618, user_id=c7dbf4f50dae470cbf2bd8f73bab4ed9, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:30.963013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:31.057300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1276/2555] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:30 2022] PUT /identity/v3/projects/61b2499065aa4c05b660c1098f1fe618/users/c7dbf4f50dae470cbf2bd8f73bab4ed9/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:31.061400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:31.071753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23451d-73b9-4ce4-8b94-574721a49853 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:31.133340 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041477a-ca83-4c80-8c9c-81936a679ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:31.133951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041477a-ca83-4c80-8c9c-81936a679ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:31.134227 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041477a-ca83-4c80-8c9c-81936a679ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:31.134494 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041477a-ca83-4c80-8c9c-81936a679ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:31.135177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041477a-ca83-4c80-8c9c-81936a679ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:31.136948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041477a-ca83-4c80-8c9c-81936a679ad0 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:31.149095 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1280/2556] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:31.157468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:31.177807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4c11a-12f5-4281-8dc3-bb762f060467 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:31.182538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:31.186206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:31.186206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:31.186206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] PATH_INFO: `/v3/projects/61b2499065aa4c05b660c1098f1fe618/users/c7dbf4f50dae470cbf2bd8f73bab4ed9/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:31.186206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b2499065aa4c05b660c1098f1fe618, user_id=c7dbf4f50dae470cbf2bd8f73bab4ed9, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:31.188323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:31.199557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1277/2557] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:31 2022] PUT /identity/v3/projects/61b2499065aa4c05b660c1098f1fe618/users/c7dbf4f50dae470cbf2bd8f73bab4ed9/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:31.215078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:31.215078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:31.215078 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:31.255539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:31.259411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] MFA Rules not processed for user `c7dbf4f50dae470cbf2bd8f73bab4ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:31.260655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:31.331548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1281/2558] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:31.342727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:31.344525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:31.404487 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a736bd-c3b1-4bb8-8f56-72fc1657b215 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:31.408213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f3e49-5b34-4d8e-b282-510b8cd6e191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:31.408808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3e49-5b34-4d8e-b282-510b8cd6e191 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:31.409115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3e49-5b34-4d8e-b282-510b8cd6e191 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:31.409385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3e49-5b34-4d8e-b282-510b8cd6e191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:31.418399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f3e49-5b34-4d8e-b282-510b8cd6e191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:31.419853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f3e49-5b34-4d8e-b282-510b8cd6e191 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:31.430991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1278/2559] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:32.634596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:32.704716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7c173-c000-4200-9525-3fe92e200c1e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:32.709721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:32.710350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:32.710624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:32.715611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] PATH_INFO: `/v3/users/31b60743a2a246dab586f3a0688f949f` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:32.715611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b60743a2a246dab586f3a0688f949f)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:32.715611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:32.981758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b60743a2a246dab586f3a0688f949f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:32.998742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b60743a2a246dab586f3a0688f949f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:32.999672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b60743a2a246dab586f3a0688f949f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:33.009683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b60743a2a246dab586f3a0688f949f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:33.127440 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b60743a2a246dab586f3a0688f949f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:33.134201 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1282/2560] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:32 2022] DELETE /identity/v3/users/31b60743a2a246dab586f3a0688f949f => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:33.141314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:33.141314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:33.141314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:33.147603 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:33.160726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:33.161981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:33.221485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1279/2561] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:33.232020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:33.233772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:33.255381 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:33.294734 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cac8c1-9c65-4015-a87f-cc18cdfa1ff3 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:33.299365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:33.300316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:33.300316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:33.300811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:33.310255 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:33.311892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:33.321906 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1283/2562] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:33.960494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:33.962200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:33.974799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e60130-1407-4a11-b4fb-72da7bf99bf0 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:33.980476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:33.980969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:33.981354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:33.981725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] PATH_INFO: `/v3/projects/2a905baf0b9c4c148f14213b0132d6fe` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:33.983460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a905baf0b9c4c148f14213b0132d6fe)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:33.984854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:34.109633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a905baf0b9c4c148f14213b0132d6fe', 'request_id': 'req-95b744dd-a99f-4909-81c6-6cc7cbc97850'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.121149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a905baf0b9c4c148f14213b0132d6fe', 'request_id': 'req-95b744dd-a99f-4909-81c6-6cc7cbc97850'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.134269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a905baf0b9c4c148f14213b0132d6fe', 'request_id': 'req-95b744dd-a99f-4909-81c6-6cc7cbc97850'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.235762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] The token cache is being invalidate because project 2a905baf0b9c4c148f14213b0132d6fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:34.236299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.236691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.238727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1280/2563] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:33 2022] DELETE /identity/v3/projects/2a905baf0b9c4c148f14213b0132d6fe => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:34.242876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:34.254603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cd3d1-d7ca-48ad-bec3-35aeac988849 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:34.319007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:34.319692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:34.319976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:34.320309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] PATH_INFO: `/v3/users/c7dbf4f50dae470cbf2bd8f73bab4ed9` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:34.321563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7dbf4f50dae470cbf2bd8f73bab4ed9)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:34.323706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:34.536908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7dbf4f50dae470cbf2bd8f73bab4ed9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.552863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7dbf4f50dae470cbf2bd8f73bab4ed9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.553936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7dbf4f50dae470cbf2bd8f73bab4ed9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.562120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7dbf4f50dae470cbf2bd8f73bab4ed9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.616888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6963d607-32a2-4b64-bc07-bd6cd484ba3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7dbf4f50dae470cbf2bd8f73bab4ed9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:34.623724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1284/2564] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:34 2022] DELETE /identity/v3/users/c7dbf4f50dae470cbf2bd8f73bab4ed9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:35.186639 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:35.189624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:35.209183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b744dd-a99f-4909-81c6-6cc7cbc97850 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:35.295660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:35.297001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.297445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.297926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] PATH_INFO: `/v3/projects/61b2499065aa4c05b660c1098f1fe618` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.300839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b2499065aa4c05b660c1098f1fe618)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:35.302956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:35.431277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:35.454528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b2499065aa4c05b660c1098f1fe618', 'request_id': 'req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:35.461241 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b2499065aa4c05b660c1098f1fe618', 'request_id': 'req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:35.467602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b2499065aa4c05b660c1098f1fe618', 'request_id': 'req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:35.549451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] The token cache is being invalidate because project 61b2499065aa4c05b660c1098f1fe618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:35.550566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:35.551434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4cdd56d9-cf95-43b1-83d6-0c351636f6a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:35.557914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1281/2565] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:35 2022] DELETE /identity/v3/projects/61b2499065aa4c05b660c1098f1fe618 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:35.559570 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.560022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.560105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.583005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:35.584728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:35.627943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:35.661066 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1285/2566] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:35.672985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.672985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.673191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.696528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:35.697990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:35.706226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1282/2567] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:35.710970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:35.721106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fc95b-e081-4df4-93ea-f889388bedb4 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:35.728530 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:35.729139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.729436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.729732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.730379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:38:35.730676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:35.732351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:35.747916 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1286/2568] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:38:35.751642 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:35.752318 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:35.754031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:35.763522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66255b85-9529-42ac-87e1-a4b51edf161b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:35.768787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:35.769785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.770060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.770324 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.771127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:35.773236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:35.830321 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fdcf0b-bff2-4591-8398-bdfb0cb521bf None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:35.836508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:35.837109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.837392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.837661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.839416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:35.851390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:35.852531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:35.862225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1283/2569] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:38:35.910241 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1287/2570] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:35 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:35.914921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:35.926017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524df56c-1109-46bc-836f-fad83f6e62a6 service placement] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:35.982550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:35.983151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:35.983456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:35.983726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:35.984341 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:35.986236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:36.117414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1284/2571] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:35 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:36.121317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:36.139130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f944456-45d2-4237-975f-4e7736fe9a50 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:36.148437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:36.148437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:36.148437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:36.148437 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:36.148897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:36.150985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:36.180216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1288/2572] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:36.181876 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:36.207710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daa943-ee00-4bec-85ae-c008ba6cd665 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:36.214062 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a6ad2-4618-4f6c-ac0e-a3bb9a68db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:36.214734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6ad2-4618-4f6c-ac0e-a3bb9a68db3a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:36.215157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6ad2-4618-4f6c-ac0e-a3bb9a68db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:36.215328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6ad2-4618-4f6c-ac0e-a3bb9a68db3a admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:36.215972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a6ad2-4618-4f6c-ac0e-a3bb9a68db3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:36.217703 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a6ad2-4618-4f6c-ac0e-a3bb9a68db3a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:36.231597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1285/2573] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:36.235564 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:36.246873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c610330c-0630-49eb-8034-598bf19888ad admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:36.251646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:36.252323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:36.252612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:36.252868 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] PATH_INFO: `/v3/projects/8b6cf82b635a47cbb31ac9dc4b28bcc3/users/b8b8e2d9737c4b839dd0b1bff1b12564/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:36.271628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6cf82b635a47cbb31ac9dc4b28bcc3, user_id=b8b8e2d9737c4b839dd0b1bff1b12564, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:36.274595 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:36.358491 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1289/2574] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:36 2022] PUT /identity/v3/projects/8b6cf82b635a47cbb31ac9dc4b28bcc3/users/b8b8e2d9737c4b839dd0b1bff1b12564/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:36.369285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce300f-dabb-4050-af9d-a98dfb519a17 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:36.369827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce300f-dabb-4050-af9d-a98dfb519a17 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:36.369827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce300f-dabb-4050-af9d-a98dfb519a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:36.401022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-84ce300f-dabb-4050-af9d-a98dfb519a17 None None] MFA Rules not processed for user `b8b8e2d9737c4b839dd0b1bff1b12564`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:36.402248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-84ce300f-dabb-4050-af9d-a98dfb519a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:36.407552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:36.472946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1286/2575] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:38:36.487731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:36.491609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:36.556759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44caaf8-2dff-46a1-bf3f-cd02cb6b227b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:36.560537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:36.561123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:36.561408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:36.561681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:36.570317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:36.571861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:36.580940 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1290/2576] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:39.361774 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:39.362251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:39.362333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:39.383921 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:39.385151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:39.451600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1287/2577] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:38:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:39.472697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:39.474618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:39.572200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cae6fe-cbdd-416d-8bbe-b1a4b19bfee7 service glance] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:39.577871 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:39.578510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:39.578801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:39.579059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:39.589590 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:39.591443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:39.600845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1291/2578] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:38:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 00:38:39.911324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:39.912954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:40.045966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3280d0b-081b-4db5-9f22-dfddadf433a1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:40.051211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:40.052092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:40.052399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:40.052816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] PATH_INFO: `/v3/users/f950ecd21b5a4612bff847604ad8efdd` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:40.054566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f950ecd21b5a4612bff847604ad8efdd)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:40.060561 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:40.272698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f950ecd21b5a4612bff847604ad8efdd'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:40.298329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:40.317078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f950ecd21b5a4612bff847604ad8efdd'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:40.317755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f950ecd21b5a4612bff847604ad8efdd'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:40.335964 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f950ecd21b5a4612bff847604ad8efdd'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:40.424158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f950ecd21b5a4612bff847604ad8efdd'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:40.434840 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1288/2579] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:39 2022] DELETE /identity/v3/users/f950ecd21b5a4612bff847604ad8efdd => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:40.529229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:40.531531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:40.554876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:40.771957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e74fd-a1f7-40c1-b247-2ca55817b95e service swift] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:40.793544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:40.796027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:40.894714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11286e6b-eaf8-4cbb-ba94-e4a2e5146f71 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:40.905358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:40.906751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:40.907734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:40.908519 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] PATH_INFO: `/v3/users/41fecf50555a4effb8aca217296bbad6` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:40.910276 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fecf50555a4effb8aca217296bbad6)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:40.913872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:40.946436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:40.947060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:40.947378 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:40.947655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:40.950259 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:40.971343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:40.974207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:40.985543 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1292/2580] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:38:40.986525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:40.988865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:41.005899 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f6321-da99-41d5-a3cf-ff7c87b4ff69 service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:41.016567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:41.017387 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:41.017832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:41.018175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] PATH_INFO: `/v3/projects/b8f2a58683be4aa486cebdf7aa8e0d83` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:41.019555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f2a58683be4aa486cebdf7aa8e0d83)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:41.021604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:41.088604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fecf50555a4effb8aca217296bbad6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.102524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fecf50555a4effb8aca217296bbad6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.103325 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fecf50555a4effb8aca217296bbad6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.115054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fecf50555a4effb8aca217296bbad6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.161763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f2a58683be4aa486cebdf7aa8e0d83', 'request_id': 'req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.170298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f2a58683be4aa486cebdf7aa8e0d83', 'request_id': 'req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.178688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f2a58683be4aa486cebdf7aa8e0d83', 'request_id': 'req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.184547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fecf50555a4effb8aca217296bbad6'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.192058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1289/2581] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:40 2022] DELETE /identity/v3/users/41fecf50555a4effb8aca217296bbad6 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:38:41.281735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] The token cache is being invalidate because project b8f2a58683be4aa486cebdf7aa8e0d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:41.282089 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.301016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.301016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1293/2582] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:40 2022] DELETE /identity/v3/projects/b8f2a58683be4aa486cebdf7aa8e0d83 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:41.545963 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:41.548057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:41.660022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:41.680674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c66f1-6e56-419e-85f4-9d4a8e7fa293 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:41.691311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:41.691311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:41.691829 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:41.692176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] PATH_INFO: `/v3/projects/fc3040045262415cb0fcb9525ba437bc` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:41.693547 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3040045262415cb0fcb9525ba437bc)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:41.695713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:41.821072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3040045262415cb0fcb9525ba437bc', 'request_id': 'req-8e74f911-865a-494f-867b-2ec05eeade9c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.827677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3040045262415cb0fcb9525ba437bc', 'request_id': 'req-8e74f911-865a-494f-867b-2ec05eeade9c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.838898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3040045262415cb0fcb9525ba437bc', 'request_id': 'req-8e74f911-865a-494f-867b-2ec05eeade9c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.915858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] The token cache is being invalidate because project fc3040045262415cb0fcb9525ba437bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:41.916654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.917174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:41.920451 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1290/2583] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:41 2022] DELETE /identity/v3/projects/fc3040045262415cb0fcb9525ba437bc => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:41.924560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:41.926730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:41.939150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:42.012194 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cd165-7444-4e6e-a3fe-1ef4e2fbe5f4 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:42.076276 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:42.076809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:42.077085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:42.077377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] PATH_INFO: `/v3/users/03110f36df2d444fad6ff04bc308be4d` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:42.078503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] RBAC: Authorizing `identity:delete_user(user_id=03110f36df2d444fad6ff04bc308be4d)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:42.080486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:42.279262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03110f36df2d444fad6ff04bc308be4d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.294152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03110f36df2d444fad6ff04bc308be4d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.294864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03110f36df2d444fad6ff04bc308be4d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.302461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03110f36df2d444fad6ff04bc308be4d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.378169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03110f36df2d444fad6ff04bc308be4d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.386206 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1294/2584] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:41 2022] DELETE /identity/v3/users/03110f36df2d444fad6ff04bc308be4d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:42.664468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:42.684813 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:42.700909 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74f911-865a-494f-867b-2ec05eeade9c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:42.792675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:42.792675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:42.793029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:42.793396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] PATH_INFO: `/v3/projects/dd8ef69b931748e689f6d2a9cd4aceaa` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:42.794936 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd8ef69b931748e689f6d2a9cd4aceaa)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:42.797272 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:42.952607 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd8ef69b931748e689f6d2a9cd4aceaa', 'request_id': 'req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.967159 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd8ef69b931748e689f6d2a9cd4aceaa', 'request_id': 'req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:42.977917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd8ef69b931748e689f6d2a9cd4aceaa', 'request_id': 'req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:43.059132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] The token cache is being invalidate because project dd8ef69b931748e689f6d2a9cd4aceaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:43.059822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:43.060674 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-1aec79e0-d208-4a8b-b24a-e11b4ef01d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:43.062700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1291/2585] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:42 2022] DELETE /identity/v3/projects/dd8ef69b931748e689f6d2a9cd4aceaa => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:52.381843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=73,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:53.056281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=79,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:55.173803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:55.175393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:55.189204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79f60a-520e-46bc-8f47-8e9fbc24480f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:55.256896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:55.257540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:55.257821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:55.258191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] PATH_INFO: `/v3/users/b8b8e2d9737c4b839dd0b1bff1b12564` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:55.259342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b8e2d9737c4b839dd0b1bff1b12564)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:55.261408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:55.458997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b8e2d9737c4b839dd0b1bff1b12564'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:55.472642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b8e2d9737c4b839dd0b1bff1b12564'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:55.473347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b8e2d9737c4b839dd0b1bff1b12564'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:55.480968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b8e2d9737c4b839dd0b1bff1b12564'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:55.753991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b8e2d9737c4b839dd0b1bff1b12564'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:55.761865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1295/2586] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:38:55 2022] DELETE /identity/v3/users/b8b8e2d9737c4b839dd0b1bff1b12564 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:55.766450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:55.766939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:55.767044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:55.786802 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:55.788041 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:55.848123 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1292/2587] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:55.859072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:55.860513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:55.918213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aaae7-f052-4e82-9677-1405287e4470 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:55.926329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6762b4-7423-4021-a53e-7999fa9114e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:55.927727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6762b4-7423-4021-a53e-7999fa9114e0 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:55.927966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6762b4-7423-4021-a53e-7999fa9114e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:55.928235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6762b4-7423-4021-a53e-7999fa9114e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:55.937527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6762b4-7423-4021-a53e-7999fa9114e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:55.938949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6762b4-7423-4021-a53e-7999fa9114e0 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:55.949352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1296/2588] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:56.479975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:56.481599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:56.498573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391ea44-d2ca-4e70-8eda-f29dbde9362f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:56.505741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:56.506537 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:56.506832 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:56.507107 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] PATH_INFO: `/v3/projects/8b6cf82b635a47cbb31ac9dc4b28bcc3` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:56.508307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6cf82b635a47cbb31ac9dc4b28bcc3)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:56.510264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:56.634499 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6cf82b635a47cbb31ac9dc4b28bcc3', 'request_id': 'req-24715219-221d-484b-b0ab-c4852fd1eb10'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:56.640625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6cf82b635a47cbb31ac9dc4b28bcc3', 'request_id': 'req-24715219-221d-484b-b0ab-c4852fd1eb10'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:56.647417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6cf82b635a47cbb31ac9dc4b28bcc3', 'request_id': 'req-24715219-221d-484b-b0ab-c4852fd1eb10'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:56.744308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] The token cache is being invalidate because project 8b6cf82b635a47cbb31ac9dc4b28bcc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:38:56.744523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:56.745157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-24715219-221d-484b-b0ab-c4852fd1eb10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:38:56.746658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1293/2589] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:38:56 2022] DELETE /identity/v3/projects/8b6cf82b635a47cbb31ac9dc4b28bcc3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:56.751986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:56.752504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:56.752612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:56.773109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:56.775936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:56.792678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:56.838020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1297/2590] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:56.845779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:56.846267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:56.846393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:56.868213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:56.869714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:56.877935 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1294/2591] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:56.882548 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:56.891988 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf34e1-5d08-40ff-9852-8ecb9645c45e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:56.897352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:56.897944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:56.899813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:56.899813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:56.899813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:38:56.899813 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:56.901136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:56.913864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1298/2592] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:38:56.918582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:56.927996 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5654b-a9b8-40e9-b6b1-e04ca84c5f32 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:56.932930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:56.933524 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:56.933800 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:56.934067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:56.934850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:56.936834 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.046868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1295/2593] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:38:56 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:38:57.050660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:57.060989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283e13-40b2-4032-aa8c-6cb15bf7d746 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:57.119770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:57.120390 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.120721 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.120942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.121674 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:57.123729 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.219950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1299/2594] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:38:57 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:38:57.225144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:57.243744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797cd48f-fa84-49fe-a299-a5f81a6a7070 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:57.249663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:57.250298 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.250573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.250859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.251665 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:57.253319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.264160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1296/2595] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:57.268995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:57.281156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ef805-bd49-478c-bcb1-b0a50065361a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:57.288030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9c8d5-790e-485d-a0d5-295b3f4ed998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:57.288030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9c8d5-790e-485d-a0d5-295b3f4ed998 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.288030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9c8d5-790e-485d-a0d5-295b3f4ed998 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.288030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9c8d5-790e-485d-a0d5-295b3f4ed998 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.288750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9c8d5-790e-485d-a0d5-295b3f4ed998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:57.290436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9c8d5-790e-485d-a0d5-295b3f4ed998 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.301788 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1300/2596] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:38:57.305889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:57.316260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccdb53-98ce-4c41-8749-7e495762deaf admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:57.321489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:57.322130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.322409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.322679 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] PATH_INFO: `/v3/projects/2aabdf53ddce4b91a70bdb6a287cb765/users/0a96c63741ad44bf946ba40b6e16d793/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.338526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aabdf53ddce4b91a70bdb6a287cb765, user_id=0a96c63741ad44bf946ba40b6e16d793, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:57.341256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.436415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1297/2597] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:38:57 2022] PUT /identity/v3/projects/2aabdf53ddce4b91a70bdb6a287cb765/users/0a96c63741ad44bf946ba40b6e16d793/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:38:57.441684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037494f6-41cd-4645-8ede-204214db46ec None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.442743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037494f6-41cd-4645-8ede-204214db46ec None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.443260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037494f6-41cd-4645-8ede-204214db46ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.462832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-037494f6-41cd-4645-8ede-204214db46ec None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:57.464127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-037494f6-41cd-4645-8ede-204214db46ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:57.529874 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1301/2598] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:38:57.539998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:57.541764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:57.564324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:57.600253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126acd-e3ff-4fed-92ac-80728e378cbb admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:57.605080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b7208-851c-4373-bcee-9e190b3a6c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:57.606030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7208-851c-4373-bcee-9e190b3a6c08 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.606773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7208-851c-4373-bcee-9e190b3a6c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.607492 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b7208-851c-4373-bcee-9e190b3a6c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.618743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b7208-851c-4373-bcee-9e190b3a6c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:57.621076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b7208-851c-4373-bcee-9e190b3a6c08 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.633966 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1298/2599] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:57.740448 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1c098-3b5b-4008-96c9-997a497ce009 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.740805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1c098-3b5b-4008-96c9-997a497ce009 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.741150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1c098-3b5b-4008-96c9-997a497ce009 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.742821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1302/2600] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:38:57.750727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b124a-7df1-4c42-af97-f31499e8d7f7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.751377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b124a-7df1-4c42-af97-f31499e8d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.751662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b124a-7df1-4c42-af97-f31499e8d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.773775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a72b124a-7df1-4c42-af97-f31499e8d7f7 None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:57.775040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a72b124a-7df1-4c42-af97-f31499e8d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:57.855191 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1299/2601] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:38:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:57.867613 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1c098-3b5b-4008-96c9-997a497ce009 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:57.869380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2ce1c098-3b5b-4008-96c9-997a497ce009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:57.902687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:38:57.931688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1c098-3b5b-4008-96c9-997a497ce009 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:57.935411 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c34108-1539-403b-ad38-b81e02f1f095 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de6c52a7640e47c7be2253e32aa5a173', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:57.936022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c34108-1539-403b-ad38-b81e02f1f095 service glance] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:57.936303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c34108-1539-403b-ad38-b81e02f1f095 service glance] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:57.936577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c34108-1539-403b-ad38-b81e02f1f095 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:57.948018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c34108-1539-403b-ad38-b81e02f1f095 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:57.949845 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c34108-1539-403b-ad38-b81e02f1f095 service glance] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:57.961719 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1303/2602] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:58.175749 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83ad9-56cd-48bd-851f-b7c5d0d47eff None None] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:58.176122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83ad9-56cd-48bd-851f-b7c5d0d47eff None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:58.176712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83ad9-56cd-48bd-851f-b7c5d0d47eff None None] PATH_INFO: `/` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:58.178452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1300/2603] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:38:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:38:58.183608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8edcbe-e8f6-4004-971c-46d23d542fb1 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:58.183921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8edcbe-e8f6-4004-971c-46d23d542fb1 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:58.184254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8edcbe-e8f6-4004-971c-46d23d542fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:58.203662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7c8edcbe-e8f6-4004-971c-46d23d542fb1 None None] MFA Rules not processed for user `6c8a8736fc89485d87f14e06e85e67c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:58.204970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7c8edcbe-e8f6-4004-971c-46d23d542fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:58.212537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1304/2604] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:38:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:58.225463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83ad9-56cd-48bd-851f-b7c5d0d47eff None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:58.240422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83ad9-56cd-48bd-851f-b7c5d0d47eff None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:58.244379 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301b8e-0f27-4913-8dec-e9c1bfeb0ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:58.245589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301b8e-0f27-4913-8dec-e9c1bfeb0ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:58.245898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301b8e-0f27-4913-8dec-e9c1bfeb0ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:58.246182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301b8e-0f27-4913-8dec-e9c1bfeb0ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:58.254030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301b8e-0f27-4913-8dec-e9c1bfeb0ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:58.255715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301b8e-0f27-4913-8dec-e9c1bfeb0ba3 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:58.263680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1301/2605] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:58.355922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624f190-0c68-4d07-a124-901fe4235976 None None] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:58.356212 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624f190-0c68-4d07-a124-901fe4235976 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:58.356490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624f190-0c68-4d07-a124-901fe4235976 None None] PATH_INFO: `/` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:58.358026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1305/2606] 10.222.0.15 () {54 vars in 921 bytes} [Mon Aug 8 00:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 00:38:58.363220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab36b3b-f8f3-437e-a88e-8af97f1858ae None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:58.363591 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab36b3b-f8f3-437e-a88e-8af97f1858ae None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:58.364046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab36b3b-f8f3-437e-a88e-8af97f1858ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:58.383666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-eab36b3b-f8f3-437e-a88e-8af97f1858ae None None] MFA Rules not processed for user `c1031699d7da4a76965035faa9305dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:58.384879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-eab36b3b-f8f3-437e-a88e-8af97f1858ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:58.394437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1302/2607] 10.222.0.15 () {58 vars in 1018 bytes} [Mon Aug 8 00:38:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:58.406015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624f190-0c68-4d07-a124-901fe4235976 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:58.408080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6624f190-0c68-4d07-a124-901fe4235976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:58.424040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624f190-0c68-4d07-a124-901fe4235976 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:58.427912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:58.428537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:58.428810 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:58.429077 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:58.440246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:58.441882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:58.453379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1306/2608] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:38:59.036582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784cf0-903b-4ece-9dec-83ada1b9b31d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:59.037072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784cf0-903b-4ece-9dec-83ada1b9b31d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:59.037180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20784cf0-903b-4ece-9dec-83ada1b9b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:59.066803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-20784cf0-903b-4ece-9dec-83ada1b9b31d None None] MFA Rules not processed for user `0a96c63741ad44bf946ba40b6e16d793`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:38:59.068919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-20784cf0-903b-4ece-9dec-83ada1b9b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:59.137659 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1303/2609] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:38:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:38:59.149449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:38:59.150954 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:38:59.209777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab62e0d-b424-4e73-8db3-dda5b9a49e18 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:38:59.213357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:38:59.213953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:38:59.214232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:38:59.214503 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:38:59.223012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:38:59.231254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:38:59.234128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1307/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:38:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:39:00.837296 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b040b-70d1-4068-8369-1ecde5f0b5b6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:00.837757 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b040b-70d1-4068-8369-1ecde5f0b5b6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:00.837863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b040b-70d1-4068-8369-1ecde5f0b5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:00.856704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7e4b040b-70d1-4068-8369-1ecde5f0b5b6 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:00.857919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7e4b040b-70d1-4068-8369-1ecde5f0b5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:00.876085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=100 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:39:00.912372 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1304/2611] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:39:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:39:00.924179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:39:00.925798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:00.980582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:00.980986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:00.981333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:00.996923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5d3ee-6531-4241-854b-f978ef2af9f8 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:39:01.001415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] MFA Rules not processed for user `0a96c63741ad44bf946ba40b6e16d793`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:01.001932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee644d49-ebf2-4510-8b95-979df86a8192 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:39:01.002524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee644d49-ebf2-4510-8b95-979df86a8192 service swift] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:01.002645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:01.002800 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee644d49-ebf2-4510-8b95-979df86a8192 service swift] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:01.003069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee644d49-ebf2-4510-8b95-979df86a8192 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:01.009902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1305/2612] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:39:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:39:01.013347 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee644d49-ebf2-4510-8b95-979df86a8192 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:39:01.015041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee644d49-ebf2-4510-8b95-979df86a8192 service swift] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:39:01.023986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1308/2613] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:39:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:39:06.877230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:39:06.878780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:06.890849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfefdf-5170-456c-9ae5-1435ccd8e06c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:39:06.894441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:39:06.895046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:06.895385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:06.895662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:06.897460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:07.088804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:39:07.090251 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:39:07.100672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1306/2614] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:39:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:39:11.019943 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=80 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:39:17.095901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=45 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:39:20.920724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63406ce-1ef6-42d2-816a-79a8e06b45e3 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:20.921565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63406ce-1ef6-42d2-816a-79a8e06b45e3 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:20.924922 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63406ce-1ef6-42d2-816a-79a8e06b45e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:20.946854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f63406ce-1ef6-42d2-816a-79a8e06b45e3 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:20.949113 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f63406ce-1ef6-42d2-816a-79a8e06b45e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:20.955551 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1309/2615] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 00:39:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:39:20.968144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:39:20.981033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d128a2-aced-4c48-a3d1-2871ffb20df0 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:39:20.986034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce64a3e-13d8-486f-8604-9f469a76a589 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:39:20.986628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce64a3e-13d8-486f-8604-9f469a76a589 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:20.986904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce64a3e-13d8-486f-8604-9f469a76a589 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:20.987199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce64a3e-13d8-486f-8604-9f469a76a589 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:20.995868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce64a3e-13d8-486f-8604-9f469a76a589 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:39:20.997450 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce64a3e-13d8-486f-8604-9f469a76a589 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:39:21.005518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1307/2616] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:39:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:39:22.988642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:22.989128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:22.989242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:23.016036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] MFA Rules not processed for user `322afd3893ac4c2ba4ab4d449f4d9159`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:23.017247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:23.086426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1310/2617] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:39:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:39:27.631333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc2f27-e143-4319-8079-d9b1f3043e33 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:27.633280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc2f27-e143-4319-8079-d9b1f3043e33 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:27.633280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc2f27-e143-4319-8079-d9b1f3043e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:27.659515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-bccc2f27-e143-4319-8079-d9b1f3043e33 None None] MFA Rules not processed for user `7b317f585f8a4da3abe24ab5cf4ee0c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:27.661089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bccc2f27-e143-4319-8079-d9b1f3043e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:27.670460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1308/2618] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:39:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:39:28.147618 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:39:28.157546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63754f96-833f-448a-982b-442c37a56bb2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:39:28.161243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:39:28.161840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:28.162168 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:28.162582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:28.164350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:28.174105 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:39:28.175675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5aa22b-4160-4a71-9680-53ccb6be5b2f service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:39:28.184948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1311/2619] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:39:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:39:35.235350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:35.235855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:35.235962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:35.255181 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] MFA Rules not processed for user `0a96c63741ad44bf946ba40b6e16d793`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:35.256458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:35.263318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1309/2620] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:39:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:39:38.181107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=34 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:39:45.249926 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=12 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:39:59.218114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33b89-728b-4588-858e-df303d6117af None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:59.220907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33b89-728b-4588-858e-df303d6117af None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:59.220907 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc33b89-728b-4588-858e-df303d6117af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:59.239110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4bc33b89-728b-4588-858e-df303d6117af None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:39:59.240358 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4bc33b89-728b-4588-858e-df303d6117af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:39:59.247445 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1312/2621] 10.222.0.15 () {58 vars in 1009 bytes} [Mon Aug 8 00:39:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:39:59.269463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:39:59.282108 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618017db-c7c2-4dc3-b7ce-49359f789c1c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:39:59.287043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:39:59.289981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:39:59.290260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:39:59.292731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:39:59.301845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:39:59.303471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:39:59.317441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1310/2622] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:39:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:40:07.249265 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc33b89-728b-4588-858e-df303d6117af None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:07.259651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc33b89-728b-4588-858e-df303d6117af None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:07.264801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:07.265577 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:07.266045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:07.266316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:07.268538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:07.282611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:07.284075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:07.294832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1313/2623] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:40:09.308257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:11.498924 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:11.500557 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:11.521578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d7b0d-5ad0-4845-8dcb-e390fee706c9 service swift] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:11.526769 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:11.527775 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:11.528281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:11.528546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] PATH_INFO: `/v3/users/fcc8eeb749a5457f98492da99199528c` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:11.530061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc8eeb749a5457f98492da99199528c)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:11.532350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:11.696408 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc8eeb749a5457f98492da99199528c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:11.714271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc8eeb749a5457f98492da99199528c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:11.715196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc8eeb749a5457f98492da99199528c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:11.723771 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc8eeb749a5457f98492da99199528c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:11.826527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ff2ad21f-bf11-4cec-a98b-c4eb39f2ba55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc8eeb749a5457f98492da99199528c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:11.833579 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1311/2624] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:40:11 2022] DELETE /identity/v3/users/fcc8eeb749a5457f98492da99199528c => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:40:12.309376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:12.319820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb4632-5b36-43b0-90fe-feb1b6259fbb service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:12.382585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:12.383219 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:12.383742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:12.383876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] PATH_INFO: `/v3/projects/1aa84c8a680d49f8be48580c62c18889` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:12.384889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aa84c8a680d49f8be48580c62c18889)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:12.386789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:12.525572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aa84c8a680d49f8be48580c62c18889', 'request_id': 'req-c2b441dc-651f-4775-a954-9e8a70a7414d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:12.535762 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aa84c8a680d49f8be48580c62c18889', 'request_id': 'req-c2b441dc-651f-4775-a954-9e8a70a7414d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:12.542895 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aa84c8a680d49f8be48580c62c18889', 'request_id': 'req-c2b441dc-651f-4775-a954-9e8a70a7414d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:12.659663 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] The token cache is being invalidate because project 1aa84c8a680d49f8be48580c62c18889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:40:12.660245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:12.660952 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-c2b441dc-651f-4775-a954-9e8a70a7414d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:12.662451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1314/2625] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:40:12 2022] DELETE /identity/v3/projects/1aa84c8a680d49f8be48580c62c18889 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:40:12.667780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:12.668827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:12.669370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:12.691483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:12.693080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:12.787954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1312/2626] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:12.791316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:12.791316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:12.791316 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:12.815191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:12.817532 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:12.831506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1315/2627] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:12.835345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:12.847848 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24422305-1fe7-4b53-bf83-4f1b63356d7f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:12.853175 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:12.853901 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:12.854285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:12.854673 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:12.855394 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:40:12.856330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:12.858183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:12.875928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1313/2628] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:40:12.877075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:12.888881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8ea2-3511-4958-8938-b07f976a7092 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:12.894262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:12.894864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:12.895132 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:12.895439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:12.896067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:12.897975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:13.007359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1316/2629] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:40:12 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:40:13.011893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:13.023480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c4727-4a66-44b7-9400-c1097643a57f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:13.097129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:13.097740 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:13.098037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:13.098304 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:13.098920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:13.100942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:13.110148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:13.212042 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1314/2630] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:40:13 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:40:13.218224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:13.229108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a77c73-bc4e-42a1-9ed2-97b72cc340a3 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:13.234204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:13.235339 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:13.235718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:13.239074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:13.239935 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:13.241777 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:13.257217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1317/2631] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:40:13.259346 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:13.270092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06f6b-3fa9-466c-920b-aefb3782d756 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:13.276759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e717a-b5c9-4758-9d9b-ebe33bd9f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:13.276759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e717a-b5c9-4758-9d9b-ebe33bd9f41e admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:13.277059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e717a-b5c9-4758-9d9b-ebe33bd9f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:13.277392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e717a-b5c9-4758-9d9b-ebe33bd9f41e admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:13.278481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e717a-b5c9-4758-9d9b-ebe33bd9f41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:13.280452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e717a-b5c9-4758-9d9b-ebe33bd9f41e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:13.295981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1315/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:40:13.297676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:13.310655 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587c0a-c06e-4ffb-84b0-e95951567490 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:13.318033 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:13.318863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:13.319398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:13.319807 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] PATH_INFO: `/v3/projects/2c27d244bd514766b955d1a4aba239eb/users/1f22b554ae524ccf9a8777ac24cdd688/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:13.339504 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c27d244bd514766b955d1a4aba239eb, user_id=1f22b554ae524ccf9a8777ac24cdd688, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:13.342444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:13.411914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1318/2633] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:40:13 2022] PUT /identity/v3/projects/2c27d244bd514766b955d1a4aba239eb/users/1f22b554ae524ccf9a8777ac24cdd688/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:40:13.417305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbe1bd-470e-41e1-b5f0-e04b3393195e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:13.419273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbe1bd-470e-41e1-b5f0-e04b3393195e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:13.419273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbe1bd-470e-41e1-b5f0-e04b3393195e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:13.443552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-84bbe1bd-470e-41e1-b5f0-e04b3393195e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:13.445136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-84bbe1bd-470e-41e1-b5f0-e04b3393195e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:13.521593 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1316/2634] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:13.530512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:13.532622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:13.544125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:13.618675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f8d5d-a71b-484b-9421-333a5d91557f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:13.625560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:13.626213 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:13.626591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:13.626728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:13.637879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:13.639798 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:13.659074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1319/2635] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:40:17.877333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad2602-2783-4d97-b9ae-507da9fd2637 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:17.877819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad2602-2783-4d97-b9ae-507da9fd2637 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:17.877930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad2602-2783-4d97-b9ae-507da9fd2637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:17.898098 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a8ad2602-2783-4d97-b9ae-507da9fd2637 None None] MFA Rules not processed for user `91ca9da53efa4421a21d5a824e25a483`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:17.899413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a8ad2602-2783-4d97-b9ae-507da9fd2637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:17.966338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1317/2636] 10.222.0.15 () {58 vars in 967 bytes} [Mon Aug 8 00:40:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:17.983867 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:17.986198 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:18.061710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e3e1-3b60-40c6-b4c6-41fea8855017 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:18.066828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaf8d31c016449e6aa6408c7f69abe20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:18.067513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:18.067736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:18.068021 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:18.077042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:18.078708 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:18.095816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1320/2637] 10.222.0.15 () {60 vars in 1384 bytes} [Mon Aug 8 00:40:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 00:40:19.336268 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:19.337324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:19.337445 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:19.373020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] MFA Rules not processed for user `1f22b554ae524ccf9a8777ac24cdd688`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:19.374737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:19.456075 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1318/2638] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:19.470459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:19.475247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:19.564498 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2b3ae-7cdd-43a5-9885-5bf65a1e9caa service swift] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:19.570042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:19.570643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:19.570923 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:19.571193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:19.579038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:19.580698 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:19.589883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1321/2639] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:40:21.249262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:21.250942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:21.271884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280f583-8eac-483f-9aa8-bdeefe47ca45 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:21.274293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:21.276878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:21.277262 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:21.277635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] PATH_INFO: `/v3/users/0a96c63741ad44bf946ba40b6e16d793` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:21.279413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a96c63741ad44bf946ba40b6e16d793)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:21.281982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:21.290184 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:21.528161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a96c63741ad44bf946ba40b6e16d793'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:21.542338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a96c63741ad44bf946ba40b6e16d793'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:21.543339 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a96c63741ad44bf946ba40b6e16d793'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:21.556029 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a96c63741ad44bf946ba40b6e16d793'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:21.650179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-ea98f8d3-b275-4ec0-930d-5a720013923d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a96c63741ad44bf946ba40b6e16d793'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:21.657604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1319/2640] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:40:21 2022] DELETE /identity/v3/users/0a96c63741ad44bf946ba40b6e16d793 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:40:21.949903 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:21.957157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:21.957157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:21.957157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:21.974304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e6d29-4a43-4ae1-b313-47e58aefb7fd service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:21.992226 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] MFA Rules not processed for user `1f22b554ae524ccf9a8777ac24cdd688`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:21.993699 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:21.997311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:22.040979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:22.041821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:22.041961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:22.042193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] PATH_INFO: `/v3/projects/2aabdf53ddce4b91a70bdb6a287cb765` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:22.045920 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aabdf53ddce4b91a70bdb6a287cb765)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:22.046044 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:22.067686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1320/2641] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:22.167984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aabdf53ddce4b91a70bdb6a287cb765', 'request_id': 'req-901344df-2088-4341-a6fc-672a9fed4266'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:22.175458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aabdf53ddce4b91a70bdb6a287cb765', 'request_id': 'req-901344df-2088-4341-a6fc-672a9fed4266'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:22.182400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aabdf53ddce4b91a70bdb6a287cb765', 'request_id': 'req-901344df-2088-4341-a6fc-672a9fed4266'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:22.273599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] The token cache is being invalidate because project 2aabdf53ddce4b91a70bdb6a287cb765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:40:22.274433 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:22.274840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:22.295670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1322/2642] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:40:21 2022] DELETE /identity/v3/projects/2aabdf53ddce4b91a70bdb6a287cb765 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:40:32.058446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:32.270298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:43.981016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:43.982472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:44.126661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb65671-6cc9-4f6e-99fd-4c63951475c2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:44.189418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:44.190031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:44.190305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:44.190573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:44.192328 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:44.204340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:44.205939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:44.216882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1321/2643] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:40:50.421298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:50.422710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:50.508780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901344df-2088-4341-a6fc-672a9fed4266 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:50.512508 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:50.513110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:50.513385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:50.513656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:50.515428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:50.604697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:50.606217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:50.625665 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1323/2644] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:40:54.198470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:54.199944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:54.217079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea369c0-9879-4fdb-b72b-23700d1f1d56 service placement] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:54.221961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:54.222583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:54.222858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:54.223129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] PATH_INFO: `/v3/users/7b317f585f8a4da3abe24ab5cf4ee0c0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:54.224261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b317f585f8a4da3abe24ab5cf4ee0c0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:54.226143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:54.246887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:54.451095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b317f585f8a4da3abe24ab5cf4ee0c0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.464118 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b317f585f8a4da3abe24ab5cf4ee0c0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.464803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b317f585f8a4da3abe24ab5cf4ee0c0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.471895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b317f585f8a4da3abe24ab5cf4ee0c0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.523516 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-db6c92d8-298c-4732-83a3-dc02df5a68c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b317f585f8a4da3abe24ab5cf4ee0c0'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.529527 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1322/2645] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:40:54 2022] DELETE /identity/v3/users/7b317f585f8a4da3abe24ab5cf4ee0c0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:40:54.777774 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:54.786919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c206f-af36-4c5a-b0ee-d6d5655b037a service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:54.842145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:54.848787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:54.849422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:54.849758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:54.850042 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] PATH_INFO: `/v3/projects/e97eb34021d44eb0a99763034c1e9e87` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:54.851375 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97eb34021d44eb0a99763034c1e9e87)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:54.853320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:54.958422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97eb34021d44eb0a99763034c1e9e87', 'request_id': 'req-3ac510c0-3077-4e12-8e40-5a4ce0429aed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.964414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97eb34021d44eb0a99763034c1e9e87', 'request_id': 'req-3ac510c0-3077-4e12-8e40-5a4ce0429aed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:54.970835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97eb34021d44eb0a99763034c1e9e87', 'request_id': 'req-3ac510c0-3077-4e12-8e40-5a4ce0429aed'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:55.089327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] The token cache is being invalidate because project e97eb34021d44eb0a99763034c1e9e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:40:55.089501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:55.090118 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-3ac510c0-3077-4e12-8e40-5a4ce0429aed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:40:55.091781 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1324/2646] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:40:54 2022] DELETE /identity/v3/projects/e97eb34021d44eb0a99763034c1e9e87 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:40:55.099468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.099468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.099468 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.155354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:55.156570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:55.214293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1323/2647] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:55.222039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.222323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.222596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.242581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:55.243834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:55.252406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1325/2648] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:55.256513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.266070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029a6c4-64b8-4948-81cb-c29c047f6660 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.270701 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.271318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.271600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.271873 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.272542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:40:55.272845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.274427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.285655 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1324/2649] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:40:55.289658 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.299454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa07a180-4e6e-4a1e-924c-0a0179bda06b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.304055 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.304661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.304937 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.305205 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.305833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.307967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.387486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1326/2650] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:40:55 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:40:55.392340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.402790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d719d1-66ac-4369-b7ee-fd9c4bfacf6f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.448791 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:55.457862 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.459748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.459748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.459748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.460046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.461778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.545764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1325/2651] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:40:55 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:40:55.549859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.560185 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699de531-9d61-46ff-8cbd-75a6dc7118ca admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.568511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.568511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.568511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.568511 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.568945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.568945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.578487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1327/2652] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:40:55.583090 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.593354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479866-8899-45bc-b72a-9272e1f6009c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.598133 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208c501-2881-4806-aa4c-bfdca15026ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.598715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208c501-2881-4806-aa4c-bfdca15026ef admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.598991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208c501-2881-4806-aa4c-bfdca15026ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.599283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208c501-2881-4806-aa4c-bfdca15026ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.599918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208c501-2881-4806-aa4c-bfdca15026ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.601469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208c501-2881-4806-aa4c-bfdca15026ef admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.612137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1326/2653] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:40:55.616317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.626856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08eec1-84e8-4a18-a5fd-3f8f11645f89 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.631497 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.632137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.632413 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.632682 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] PATH_INFO: `/v3/projects/c56adfa2f0bf4dd2ba3dbd0db883009f/users/c9f42cb4a8a643c58a9312bbcaef42de/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.647778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56adfa2f0bf4dd2ba3dbd0db883009f, user_id=c9f42cb4a8a643c58a9312bbcaef42de, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.650360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.707303 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1328/2654] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:40:55 2022] PUT /identity/v3/projects/c56adfa2f0bf4dd2ba3dbd0db883009f/users/c9f42cb4a8a643c58a9312bbcaef42de/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:40:55.721977 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.722324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.722594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.749595 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] MFA Rules not processed for user `c9f42cb4a8a643c58a9312bbcaef42de`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:40:55.751436 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:55.815544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1327/2655] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:40:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:40:55.827315 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:55.829621 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:55.916286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:40:55.930661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842029-f573-4dab-8eb9-c7607213852f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:55.936385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:55.936983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:55.937262 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:55.937536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:55.946374 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:40:55.948742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:40:55.957730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1329/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:40:59.910280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:40:59.911725 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:40:59.968942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1c285-4cad-4784-923f-f44ebdd0373d None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:40:59.973306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:40:59.973932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:40:59.974206 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:40:59.974498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:40:59.976237 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:00.036009 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:00.037433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0ad50-189d-4644-8043-aa8d114b2790 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:00.046819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1328/2657] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:40:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:05.250837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:05.313175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c0b03-f888-43a7-b0ff-13b2fa208656 service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:05.317875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:05.318493 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:05.318773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:05.319048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] PATH_INFO: `/v3/users/c9f42cb4a8a643c58a9312bbcaef42de` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:05.320197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f42cb4a8a643c58a9312bbcaef42de)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:05.322172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:05.500554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f42cb4a8a643c58a9312bbcaef42de'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:05.512169 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f42cb4a8a643c58a9312bbcaef42de'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:05.512818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f42cb4a8a643c58a9312bbcaef42de'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:05.519557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f42cb4a8a643c58a9312bbcaef42de'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:05.579510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f42cb4a8a643c58a9312bbcaef42de'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:05.586753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1330/2658] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:05 2022] DELETE /identity/v3/users/c9f42cb4a8a643c58a9312bbcaef42de => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:05.594987 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:05.595490 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:05.595763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:05.614866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:05.616091 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:05.625242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:05.675522 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1329/2659] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:05.685469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:05.686882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:05.745226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d1b7d-515b-42d8-83b4-8446b9f26c0a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:05.748866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:05.749458 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:05.749764 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:05.750036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:05.759067 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:05.760525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:05.770756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1331/2660] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:41:06.153526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.154958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:06.171369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e930a-e02d-4039-b5d5-dca8f94c5464 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.177717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.184794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.184794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.184794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] PATH_INFO: `/v3/projects/c56adfa2f0bf4dd2ba3dbd0db883009f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.184794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56adfa2f0bf4dd2ba3dbd0db883009f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.184794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.309102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56adfa2f0bf4dd2ba3dbd0db883009f', 'request_id': 'req-a00892eb-5e91-4d16-92a7-4bff5efc0fde'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:06.313383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.315273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56adfa2f0bf4dd2ba3dbd0db883009f', 'request_id': 'req-a00892eb-5e91-4d16-92a7-4bff5efc0fde'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:06.315416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:06.323017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56adfa2f0bf4dd2ba3dbd0db883009f', 'request_id': 'req-a00892eb-5e91-4d16-92a7-4bff5efc0fde'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:06.324822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:06.328823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77e987-6a19-4075-9a7c-014e94ad8885 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.374419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] The token cache is being invalidate because project c56adfa2f0bf4dd2ba3dbd0db883009f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:06.374715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:06.375392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-a00892eb-5e91-4d16-92a7-4bff5efc0fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:06.376928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1330/2661] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:06 2022] DELETE /identity/v3/projects/c56adfa2f0bf4dd2ba3dbd0db883009f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:06.381023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.381846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.382122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.382391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.382525 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.382676 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.382953 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.384295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:06.404095 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:06.405233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:06.450619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.452330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec7cc2-35a7-401c-8ea6-2689e05601bd service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.462524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1332/2662] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:41:06.467097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1331/2663] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:06.474794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.475072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.475397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.495173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:06.496410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:06.505008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1333/2664] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:06.509381 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.519382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acef4f17-91b4-4f15-92d5-8f5834cf25be None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.524734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.525447 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.525726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.525992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.526633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:41:06.526930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.528546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.541440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1332/2665] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:41:06.545404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.555735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aded36-192c-4035-8f0c-4f3bafa8c57d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.560429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.561028 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.561304 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.561573 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.562204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.564145 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.679023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1334/2666] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:06 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:06.683370 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.693733 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248fd3e-a4b9-4a34-a6c4-e82db1780806 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.717633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:06.752257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.752958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.753135 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.753428 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.754053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.756022 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.875355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1333/2667] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:06 2022] POST /identity/v3/users => generated 284 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:06.876840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.887585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665e3ae-59f4-4c29-9c76-1a26a92b1831 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.892275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.892872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.893173 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.893477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.894250 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.895905 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.905983 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1335/2668] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:06.910836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.920973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212de8d-28a4-4c31-a3bf-ac7802b6ab4b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.926470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850f0c1-d979-41a7-acc5-36122255d096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.926997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850f0c1-d979-41a7-acc5-36122255d096 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.927315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850f0c1-d979-41a7-acc5-36122255d096 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.927696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850f0c1-d979-41a7-acc5-36122255d096 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.928333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850f0c1-d979-41a7-acc5-36122255d096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.929895 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b850f0c1-d979-41a7-acc5-36122255d096 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:06.943261 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1334/2669] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:41:06.944439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:06.954457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2d987-25fc-4e74-98f3-2e1066f5bcdc admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:06.959609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:06.960267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:06.960539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:06.961155 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] PATH_INFO: `/v3/projects/6326f0f596a84ed3b60c9460b5e2831a/users/a52615eb07814584984811532383414a/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:06.976691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=6326f0f596a84ed3b60c9460b5e2831a, user_id=a52615eb07814584984811532383414a, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:06.979396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.060859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1336/2670] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:06 2022] PUT /identity/v3/projects/6326f0f596a84ed3b60c9460b5e2831a/users/a52615eb07814584984811532383414a/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:07.073696 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.074068 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.074366 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.104685 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] MFA Rules not processed for user `a52615eb07814584984811532383414a`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:07.105962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:07.173750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1335/2671] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:07.174921 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:07.229199 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:07.241369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57693bf-93db-494c-8c5d-dbe4736cd4ad admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:07.246128 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:07.246863 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.247142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.247546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.248229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:07.250117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.405085 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1337/2672] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:07 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:07.410715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:07.428371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258b55e-09bd-4395-8c34-34f58612878c None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:07.499016 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:07.500444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.500898 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.501176 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.501819 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:07.504764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.621094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1336/2673] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:07 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:07.625191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:07.636013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d291f79-f931-4151-941d-9d4299e6d50e admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:07.640533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:07.641124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.641396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.641691 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.642323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:07.643938 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.655009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1338/2674] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:41:07.659131 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:07.669496 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00627ffd-0320-40c2-827e-350346b36519 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:07.674127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:07.674779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.675056 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.675355 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] PATH_INFO: `/v3/projects/59739779ce7a40d4b88370852e97efdd/users/e58479b44ee74615955c385cea939b5b/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.690730 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59739779ce7a40d4b88370852e97efdd, user_id=e58479b44ee74615955c385cea939b5b, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:07.693868 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.771872 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:07.776709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1337/2675] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:07 2022] PUT /identity/v3/projects/59739779ce7a40d4b88370852e97efdd/users/e58479b44ee74615955c385cea939b5b/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:07.780484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:07.791348 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92524935-505f-4c68-b6ea-a2d175bcfa0f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:07.849608 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94aa23-9240-4d54-b257-11bc34e09383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:07.850217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94aa23-9240-4d54-b257-11bc34e09383 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.850495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94aa23-9240-4d54-b257-11bc34e09383 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.850818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94aa23-9240-4d54-b257-11bc34e09383 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.851438 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94aa23-9240-4d54-b257-11bc34e09383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:07.853032 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94aa23-9240-4d54-b257-11bc34e09383 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.863466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1339/2676] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:07.867435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:07.877576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfc2fb-1948-464b-ad7a-9c9e59019643 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:07.882932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:07.883692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.883971 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.884242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] PATH_INFO: `/v3/projects/59739779ce7a40d4b88370852e97efdd/users/e58479b44ee74615955c385cea939b5b/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.886163 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59739779ce7a40d4b88370852e97efdd, user_id=e58479b44ee74615955c385cea939b5b, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:07.888827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:07.899266 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1338/2677] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:07 2022] PUT /identity/v3/projects/59739779ce7a40d4b88370852e97efdd/users/e58479b44ee74615955c385cea939b5b/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:07.909402 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:07.916191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:07.916191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:07.940604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] MFA Rules not processed for user `e58479b44ee74615955c385cea939b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:07.942849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:08.025951 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1340/2678] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:08.034357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:08.036410 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:08.121183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48c429-939c-41da-a31f-3554184f907b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:08.125333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:08.126070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:08.126414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:08.126693 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:08.137658 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:08.139202 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:08.150052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1339/2679] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:08.255294 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:08.256902 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:08.270887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28247e0-aee3-41d1-afe9-2adbe69469a2 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:08.275018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6ace6-0dda-46d9-bf21-24b92f3156ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:08.275696 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6ace6-0dda-46d9-bf21-24b92f3156ed service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:08.275978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6ace6-0dda-46d9-bf21-24b92f3156ed service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:08.276252 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6ace6-0dda-46d9-bf21-24b92f3156ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:08.330555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:08.345342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6ace6-0dda-46d9-bf21-24b92f3156ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:08.346790 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6ace6-0dda-46d9-bf21-24b92f3156ed service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:08.358023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1341/2680] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:13.163250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:13.242597 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee6ce0-0b5d-469a-9473-cef7934661ff service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:13.247583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:13.252385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:13.252385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:13.252385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] PATH_INFO: `/v3/users/a52615eb07814584984811532383414a` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:13.252385 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52615eb07814584984811532383414a)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:13.252836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:13.623610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52615eb07814584984811532383414a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:13.638380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52615eb07814584984811532383414a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:13.639040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52615eb07814584984811532383414a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:13.644454 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:13.646500 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52615eb07814584984811532383414a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:13.756806 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52615eb07814584984811532383414a'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:13.763192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1340/2681] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:13 2022] DELETE /identity/v3/users/a52615eb07814584984811532383414a => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:13.767745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:13.768216 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:13.768327 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:13.787639 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:13.789976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:13.800777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:13.803318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:13.854624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1342/2682] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:13.865186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:13.866859 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:13.876994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6481c5c-4bee-47a7-9c91-f942d97c4e0b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:13.881590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:13.882216 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:13.882494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:13.882758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] PATH_INFO: `/v3/users/0797d03452e9419d93e79ccf1c222391` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:13.883928 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0797d03452e9419d93e79ccf1c222391)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:13.885814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:13.923843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b15fd-adeb-492d-a203-dad0a0ecb9e6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:13.928156 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:13.928750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:13.929025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:13.929293 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:13.938310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:13.939878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:13.949424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1343/2683] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:41:14.073479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0797d03452e9419d93e79ccf1c222391'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.085337 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0797d03452e9419d93e79ccf1c222391'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.086017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0797d03452e9419d93e79ccf1c222391'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.092748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0797d03452e9419d93e79ccf1c222391'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.163744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0797d03452e9419d93e79ccf1c222391'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.174486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1341/2684] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:13 2022] DELETE /identity/v3/users/0797d03452e9419d93e79ccf1c222391 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:14.337272 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:14.338864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:14.353840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de74d-0f8f-479c-9d59-b14f408d626e service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:14.393551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:14.395575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:14.414566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:14.415987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:14.415987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:14.415987 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] PATH_INFO: `/v3/projects/6326f0f596a84ed3b60c9460b5e2831a` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:14.417020 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6326f0f596a84ed3b60c9460b5e2831a)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:14.419311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:14.471558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a8b2f-9685-449c-afac-8680b23ce234 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:14.476697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:14.477315 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:14.477589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:14.477856 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] PATH_INFO: `/v3/projects/3101620baddd42c48179a6f411bbd33e` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:14.478950 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3101620baddd42c48179a6f411bbd33e)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:14.480861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:14.488320 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:14.491368 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:14.501148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6326f0f596a84ed3b60c9460b5e2831a', 'request_id': 'req-02f83378-2595-4944-8827-5f52c1950e70'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.506968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6326f0f596a84ed3b60c9460b5e2831a', 'request_id': 'req-02f83378-2595-4944-8827-5f52c1950e70'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.513243 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6326f0f596a84ed3b60c9460b5e2831a', 'request_id': 'req-02f83378-2595-4944-8827-5f52c1950e70'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.581960 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3101620baddd42c48179a6f411bbd33e', 'request_id': 'req-93fca06a-9d90-4eed-8a86-c532ff6607a7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.588079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3101620baddd42c48179a6f411bbd33e', 'request_id': 'req-93fca06a-9d90-4eed-8a86-c532ff6607a7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.589898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] The token cache is being invalidate because project 6326f0f596a84ed3b60c9460b5e2831a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:14.590214 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.590835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.594837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3101620baddd42c48179a6f411bbd33e', 'request_id': 'req-93fca06a-9d90-4eed-8a86-c532ff6607a7'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.595811 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1344/2685] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:14 2022] DELETE /identity/v3/projects/6326f0f596a84ed3b60c9460b5e2831a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:14.596642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:14.598200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:14.648633 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] The token cache is being invalidate because project 3101620baddd42c48179a6f411bbd33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:14.648859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.649530 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.651335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1342/2686] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:14 2022] DELETE /identity/v3/projects/3101620baddd42c48179a6f411bbd33e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:14.655835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:14.657154 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:14.667622 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f83378-2595-4944-8827-5f52c1950e70 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:14.732060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fca06a-9d90-4eed-8a86-c532ff6607a7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:14.734152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:14.734979 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:14.735281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:14.735556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] PATH_INFO: `/v3/users/e58479b44ee74615955c385cea939b5b` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:14.736900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58479b44ee74615955c385cea939b5b)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:14.737225 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:14.737846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:14.738120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:14.738709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] PATH_INFO: `/v3/users/4f21395607594c5dad1c0d1b13ad17cf` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:14.738842 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:14.740084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f21395607594c5dad1c0d1b13ad17cf)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:14.741986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:14.902485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f21395607594c5dad1c0d1b13ad17cf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.915293 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f21395607594c5dad1c0d1b13ad17cf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.915998 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f21395607594c5dad1c0d1b13ad17cf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.919478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58479b44ee74615955c385cea939b5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.923670 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f21395607594c5dad1c0d1b13ad17cf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.932534 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58479b44ee74615955c385cea939b5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.934966 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58479b44ee74615955c385cea939b5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.935540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:14.940711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58479b44ee74615955c385cea939b5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.962885 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f21395607594c5dad1c0d1b13ad17cf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:14.969138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1343/2687] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:14 2022] DELETE /identity/v3/users/4f21395607594c5dad1c0d1b13ad17cf => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:15.009091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58479b44ee74615955c385cea939b5b'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.016097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1345/2688] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:14 2022] DELETE /identity/v3/users/e58479b44ee74615955c385cea939b5b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:15.226962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:15.232569 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:15.254942 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:15.305297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a02de-8888-4eae-b2d0-c3cf90aaffb7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:15.361406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:15.362034 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:15.362312 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:15.362588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] PATH_INFO: `/v3/projects/eea2a896babe4ab2b7136af9bb4c3851` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:15.364295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea2a896babe4ab2b7136af9bb4c3851)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:15.366174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:15.422242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:15.423724 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:15.438483 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef516d2-3109-4ab7-9af7-27d833ec0665 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:15.459709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea2a896babe4ab2b7136af9bb4c3851', 'request_id': 'req-45d348be-e72a-4c82-b11b-97c8d350f115'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.465508 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea2a896babe4ab2b7136af9bb4c3851', 'request_id': 'req-45d348be-e72a-4c82-b11b-97c8d350f115'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.471851 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea2a896babe4ab2b7136af9bb4c3851', 'request_id': 'req-45d348be-e72a-4c82-b11b-97c8d350f115'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.506126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:15.506757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:15.507064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:15.507377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] PATH_INFO: `/v3/projects/59739779ce7a40d4b88370852e97efdd` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:15.508512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59739779ce7a40d4b88370852e97efdd)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:15.510462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:15.597066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] The token cache is being invalidate because project eea2a896babe4ab2b7136af9bb4c3851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:15.597357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.598094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-45d348be-e72a-4c82-b11b-97c8d350f115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.599657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1344/2689] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:15 2022] DELETE /identity/v3/projects/eea2a896babe4ab2b7136af9bb4c3851 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:15.653978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59739779ce7a40d4b88370852e97efdd', 'request_id': 'req-0adc99ff-8f32-4165-a567-de604c92ae25'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.669789 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59739779ce7a40d4b88370852e97efdd', 'request_id': 'req-0adc99ff-8f32-4165-a567-de604c92ae25'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.679785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59739779ce7a40d4b88370852e97efdd', 'request_id': 'req-0adc99ff-8f32-4165-a567-de604c92ae25'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.735277 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] The token cache is being invalidate because project 59739779ce7a40d4b88370852e97efdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:15.735636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.736398 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0adc99ff-8f32-4165-a567-de604c92ae25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:15.738053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1346/2690] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:15 2022] DELETE /identity/v3/projects/59739779ce7a40d4b88370852e97efdd => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:15.751219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:15.751721 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:15.752085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:15.777664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:15.779072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:15.835192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:15.847961 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1345/2691] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:15.856469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:15.856748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:15.857025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:15.877405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:15.878364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:15.886767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1347/2692] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:15.891342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:15.900803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f380f1e-57e3-4385-8274-69d854d562a4 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:15.905628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:15.906259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:15.906602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:15.906866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:15.907551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:41:15.907857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:15.909438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:15.921013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1346/2693] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:41:15.928581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:15.938537 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e646593-3a42-414e-98fb-5b553d79e57b None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:15.944038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:15.944636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:15.944963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:15.945236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:15.945888 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:15.947904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.032108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1348/2694] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:15 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:16.036812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.046864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b0b3-5547-41f4-aa26-7aeb9b93d859 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.102954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.103578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.103859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.104127 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.104736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.106613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.179125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1347/2695] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:16 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:41:16.183254 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.194292 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba6979-fa8e-4423-80ba-0654231053b3 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.198946 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.199568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.199848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.200116 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.200750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.202325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.213101 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1349/2696] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:16.216972 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.226833 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014f7e-20d8-4119-a30b-105ab5a61a3a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.231412 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e6e0f-3438-4956-8a96-41f4dc061ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.231994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6e0f-3438-4956-8a96-41f4dc061ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.232252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6e0f-3438-4956-8a96-41f4dc061ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.232521 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e6e0f-3438-4956-8a96-41f4dc061ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.233146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6e0f-3438-4956-8a96-41f4dc061ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.234737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e6e0f-3438-4956-8a96-41f4dc061ef8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.245558 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1348/2697] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:16.249711 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.260413 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c71a5d-7faa-4064-9c71-3dba40e31bf3 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.265112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.265744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.266016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.266281 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] PATH_INFO: `/v3/projects/6bcf6c54fcdf42bc910c854bd33a3b2c/users/9855a13e0c91461a9a13c225e5dab727/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.281231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcf6c54fcdf42bc910c854bd33a3b2c, user_id=9855a13e0c91461a9a13c225e5dab727, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.283944 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.369616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1350/2698] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:16 2022] PUT /identity/v3/projects/6bcf6c54fcdf42bc910c854bd33a3b2c/users/9855a13e0c91461a9a13c225e5dab727/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:16.381422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.381707 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.381978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.410510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] MFA Rules not processed for user `9855a13e0c91461a9a13c225e5dab727`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:16.411782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:16.475390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1349/2699] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:16.480010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.493103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:16.542172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437fbef-9ce4-4799-ac2b-ba8f084d1a56 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.546834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.547467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.547747 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.548019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.548651 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.550582 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.643459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1351/2700] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:16 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:41:16.647618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.658618 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376b9b0-b205-4e8c-890d-2bb02d7d4b41 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.704550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:16.716210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.717902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.717902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.717902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.718195 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.719891 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.791278 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1350/2701] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:16 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:16.795290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.806197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd6240-fb0d-41a9-b54e-bfb95397009b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.810818 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.811461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.811741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.812013 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.812647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.814266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.824915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1352/2702] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:16.829093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.839307 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa0830-5e65-43a2-ab72-32a36f7287a2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.844058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.844698 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.845001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.845277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] PATH_INFO: `/v3/projects/dee774e1f98f411aabe207580d9b206f/users/d553afaf2a0441429d6c834cf4b329bc/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.860640 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee774e1f98f411aabe207580d9b206f, user_id=d553afaf2a0441429d6c834cf4b329bc, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.863333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.911864 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1351/2703] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:16 2022] PUT /identity/v3/projects/dee774e1f98f411aabe207580d9b206f/users/d553afaf2a0441429d6c834cf4b329bc/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:16.915889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:16.926586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e1f46-3b49-4eb7-b702-0e24f2866e24 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:16.980393 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55eb707-a21c-4b9e-a921-2c12d96cd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:16.981015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eb707-a21c-4b9e-a921-2c12d96cd948 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:16.981245 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eb707-a21c-4b9e-a921-2c12d96cd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:16.981512 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eb707-a21c-4b9e-a921-2c12d96cd948 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:16.982134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55eb707-a21c-4b9e-a921-2c12d96cd948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:16.983733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55eb707-a21c-4b9e-a921-2c12d96cd948 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:16.993683 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1353/2704] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:16.997797 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:17.007857 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac0d7a-3df0-4419-8205-092ab169a400 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:17.012407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:17.013043 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:17.013314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:17.013581 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] PATH_INFO: `/v3/projects/dee774e1f98f411aabe207580d9b206f/users/d553afaf2a0441429d6c834cf4b329bc/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:17.015471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee774e1f98f411aabe207580d9b206f, user_id=d553afaf2a0441429d6c834cf4b329bc, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:17.018047 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:17.028145 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1352/2705] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:16 2022] PUT /identity/v3/projects/dee774e1f98f411aabe207580d9b206f/users/d553afaf2a0441429d6c834cf4b329bc/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:17.039400 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:17.039702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:17.039970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:17.067040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] MFA Rules not processed for user `d553afaf2a0441429d6c834cf4b329bc`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:17.068186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:17.132969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1354/2706] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:17.143849 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:17.146004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:17.200980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa956ad-b1fe-45e0-ac1c-951726a9e445 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:17.204627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:17.205212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:17.205489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:17.205761 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:17.215374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:17.216781 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:17.226623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1353/2707] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:17.326575 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:17.337030 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8743fa4-bf2d-41a8-8a52-116adfa36a8d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:17.340602 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8953f0e-d2b0-4fc6-a69e-f17d302b8645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:17.341197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8953f0e-d2b0-4fc6-a69e-f17d302b8645 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:17.341471 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8953f0e-d2b0-4fc6-a69e-f17d302b8645 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:17.341741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8953f0e-d2b0-4fc6-a69e-f17d302b8645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:17.345750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:17.399069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8953f0e-d2b0-4fc6-a69e-f17d302b8645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:17.400638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8953f0e-d2b0-4fc6-a69e-f17d302b8645 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:17.409817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1355/2708] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:20.432515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:20.493533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031bec0-1d6f-4ffb-b996-8b8bfdc53a77 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:20.498061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:20.498677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:20.498948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:20.499219 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] PATH_INFO: `/v3/users/9855a13e0c91461a9a13c225e5dab727` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:20.500369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9855a13e0c91461a9a13c225e5dab727)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:20.502252 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:20.739144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9855a13e0c91461a9a13c225e5dab727'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:20.740434 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:20.751255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9855a13e0c91461a9a13c225e5dab727'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:20.751930 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9855a13e0c91461a9a13c225e5dab727'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:20.758886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9855a13e0c91461a9a13c225e5dab727'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:20.837456 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9855a13e0c91461a9a13c225e5dab727'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:20.843903 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1354/2709] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:20 2022] DELETE /identity/v3/users/9855a13e0c91461a9a13c225e5dab727 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:20.848370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:20.848840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:20.848968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:20.870657 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:20.872288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:20.931469 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1356/2710] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:41:20.941010 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:20.942437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:20.998743 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43b337-356b-4b68-ac44-75d78642398f admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:21.002349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:21.002937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:21.003209 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:21.003504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:21.012826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:21.014230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:21.024446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1355/2711] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:21.455947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:21.457476 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:21.470351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22193b-146d-4d91-8579-54f2176fe31e None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:21.476040 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:21.476963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:21.479189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:21.479189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] PATH_INFO: `/v3/projects/6bcf6c54fcdf42bc910c854bd33a3b2c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:21.479189 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcf6c54fcdf42bc910c854bd33a3b2c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:21.480579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:21.649083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcf6c54fcdf42bc910c854bd33a3b2c', 'request_id': 'req-1222468e-d227-4243-a1c4-c84faabd4ef9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.654684 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcf6c54fcdf42bc910c854bd33a3b2c', 'request_id': 'req-1222468e-d227-4243-a1c4-c84faabd4ef9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.660835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcf6c54fcdf42bc910c854bd33a3b2c', 'request_id': 'req-1222468e-d227-4243-a1c4-c84faabd4ef9'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.733163 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] The token cache is being invalidate because project 6bcf6c54fcdf42bc910c854bd33a3b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:21.733397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.734012 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.735510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1357/2712] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:21 2022] DELETE /identity/v3/projects/6bcf6c54fcdf42bc910c854bd33a3b2c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:21.739462 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:21.749738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55ab02-3b68-454d-a98a-a153573b610e service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:21.804439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:21.805072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:21.805351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:21.805660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] PATH_INFO: `/v3/users/d553afaf2a0441429d6c834cf4b329bc` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:21.806724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d553afaf2a0441429d6c834cf4b329bc)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:21.808651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:21.973116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd553afaf2a0441429d6c834cf4b329bc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.985865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd553afaf2a0441429d6c834cf4b329bc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.986489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd553afaf2a0441429d6c834cf4b329bc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:21.991576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:21.995333 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd553afaf2a0441429d6c834cf4b329bc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.066117 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2a3fba78-ee41-4406-8c49-140becb05c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd553afaf2a0441429d6c834cf4b329bc'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.072470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1356/2713] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:21 2022] DELETE /identity/v3/users/d553afaf2a0441429d6c834cf4b329bc => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:22.526071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:22.528578 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:22.542353 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222468e-d227-4243-a1c4-c84faabd4ef9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:22.571846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:22.602809 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:22.603473 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:22.603748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:22.604016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] PATH_INFO: `/v3/projects/dee774e1f98f411aabe207580d9b206f` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:22.605133 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee774e1f98f411aabe207580d9b206f)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:22.607031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:22.712945 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee774e1f98f411aabe207580d9b206f', 'request_id': 'req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.718408 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee774e1f98f411aabe207580d9b206f', 'request_id': 'req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.724576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee774e1f98f411aabe207580d9b206f', 'request_id': 'req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.797275 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] The token cache is being invalidate because project dee774e1f98f411aabe207580d9b206f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:22.797560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.798186 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-43a165ce-51dd-47a9-a0c2-803ce3ccc7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:22.799780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1358/2714] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:22 2022] DELETE /identity/v3/projects/dee774e1f98f411aabe207580d9b206f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:22.805734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:22.806533 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:22.806720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:22.827055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:22.828433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:22.888064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1357/2715] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:41:22.895757 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840445bc-0d57-4d55-b098-9164a378043a None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:22.896046 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840445bc-0d57-4d55-b098-9164a378043a None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:22.896321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840445bc-0d57-4d55-b098-9164a378043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:22.916266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-840445bc-0d57-4d55-b098-9164a378043a None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:22.917470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-840445bc-0d57-4d55-b098-9164a378043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:22.925282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1359/2716] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:41:22.929918 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:22.939459 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360e9c5-2ed7-4660-97b2-499e5d87cf71 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:22.944712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:22.945301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:22.945576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:22.945846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:22.946485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:41:22.946783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:22.948406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:22.960051 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1358/2717] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:41:22.964330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840445bc-0d57-4d55-b098-9164a378043a None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:22.973634 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840445bc-0d57-4d55-b098-9164a378043a None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:22.978519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:22.979107 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:22.979416 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:22.979688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:22.980314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:22.982282 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.136930 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1360/2718] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:22 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:23.141182 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.150958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e30bb-d166-4ee8-87f8-5c72dc056a42 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.205878 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.206513 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.206795 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.207078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.207975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.210194 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.316908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1359/2719] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:23 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:23.321191 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.331317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be92bb-20ec-4c70-b346-7a6d525e2554 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.336218 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.336805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.337082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.337354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.337985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.339625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.349882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1361/2720] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:23.354501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.364397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb9b4c-7c91-4165-8a62-08f47a47c45b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.369614 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9a4ff-4fb0-4cb3-98f3-55bc507cdbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.370201 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9a4ff-4fb0-4cb3-98f3-55bc507cdbfe admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.370476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9a4ff-4fb0-4cb3-98f3-55bc507cdbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.370747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9a4ff-4fb0-4cb3-98f3-55bc507cdbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.371407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9a4ff-4fb0-4cb3-98f3-55bc507cdbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.372981 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9a4ff-4fb0-4cb3-98f3-55bc507cdbfe admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.383360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1360/2721] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:23.387533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.397267 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c36824-3161-45e6-9428-b77e8e565070 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.401828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.402460 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.402736 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.403006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] PATH_INFO: `/v3/projects/91304bd229c440aa95f52f2ae46f6a74/users/3f7af1a59c77403fb32698dd243a17ef/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.417664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91304bd229c440aa95f52f2ae46f6a74, user_id=3f7af1a59c77403fb32698dd243a17ef, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.420332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.495495 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1362/2722] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:23 2022] PUT /identity/v3/projects/91304bd229c440aa95f52f2ae46f6a74/users/3f7af1a59c77403fb32698dd243a17ef/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:23.506827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.507115 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.507422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.535716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] MFA Rules not processed for user `3f7af1a59c77403fb32698dd243a17ef`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:23.537783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:23.580931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:23.604409 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1361/2723] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:23.609441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.670334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:23.675780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17335d3a-8d54-4e81-8663-4d755872ba76 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.680365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.680995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.681274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.681559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.682166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.684099 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.781787 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1363/2724] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:23 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:23.785799 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.797275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5632ca-73b8-4bf7-9b14-18ee4348159f None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.856297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.857017 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.857170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.857443 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.858055 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.860048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.957609 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1362/2725] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:23 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:23.962873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:23.973152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed0299-6504-49e0-a0d1-ff7c496d9e90 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:23.978061 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:23.978672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:23.978964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:23.981522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:23.981522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:23.982697 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:23.993648 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1364/2726] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:23.998040 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:24.009888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b0dc8-d83b-45db-86f7-91b1633f6b64 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:24.014758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:24.015541 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:24.015814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:24.016111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] PATH_INFO: `/v3/projects/46a05029560a40f497793d95775512bd/users/4f67c6a5f1f744d7bcc02d9dd9bb7fbf/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:24.031612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a05029560a40f497793d95775512bd, user_id=4f67c6a5f1f744d7bcc02d9dd9bb7fbf, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:24.034711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:24.089937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1363/2727] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:23 2022] PUT /identity/v3/projects/46a05029560a40f497793d95775512bd/users/4f67c6a5f1f744d7bcc02d9dd9bb7fbf/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:24.093985 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:24.104342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdc09e-fccd-4c0b-acf9-14875b8347ee admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:24.161296 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab454c9f-fe86-4b11-93e4-24e8520cf3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:24.162119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab454c9f-fe86-4b11-93e4-24e8520cf3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:24.162403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab454c9f-fe86-4b11-93e4-24e8520cf3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:24.162676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab454c9f-fe86-4b11-93e4-24e8520cf3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:24.163383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab454c9f-fe86-4b11-93e4-24e8520cf3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:24.164976 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab454c9f-fe86-4b11-93e4-24e8520cf3e9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:24.174647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1365/2728] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:24.178949 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:24.189168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2edc22-7b1b-4aff-8433-f55894e66750 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:24.194406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:24.195045 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:24.195360 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:24.195632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] PATH_INFO: `/v3/projects/46a05029560a40f497793d95775512bd/users/4f67c6a5f1f744d7bcc02d9dd9bb7fbf/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:24.197586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a05029560a40f497793d95775512bd, user_id=4f67c6a5f1f744d7bcc02d9dd9bb7fbf, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:24.200255 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:24.211199 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1364/2729] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:24 2022] PUT /identity/v3/projects/46a05029560a40f497793d95775512bd/users/4f67c6a5f1f744d7bcc02d9dd9bb7fbf/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:24.221436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:24.221802 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:24.222115 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:24.253986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] MFA Rules not processed for user `4f67c6a5f1f744d7bcc02d9dd9bb7fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:24.255264 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:24.325834 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1366/2730] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:24 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:24.336951 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:24.338638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:24.396625 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c497a3-4c68-4948-ba1c-2156939d34f9 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:24.400277 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:24.400877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:24.401156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:24.401427 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:24.410933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:24.412622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:24.422310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1365/2731] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:24 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:41:24.447356 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:24.457193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22f70d-d08b-4fa9-ac74-8f2ea44cb5eb None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:24.461360 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:24.461891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:24.462166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:24.462434 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:24.464165 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:24.512771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:24.523593 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:24.525019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697869a-9fd2-422b-9fdc-c9c32ea7cc48 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:24.533831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1367/2732] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:25.088479 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:25.129910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:25.151013 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a175f9f-a5a3-47c4-a81e-3bb038d8b600 service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:25.155548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:25.156168 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:25.156455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:25.156717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] PATH_INFO: `/v3/users/3f7af1a59c77403fb32698dd243a17ef` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:25.157822 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f7af1a59c77403fb32698dd243a17ef)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:25.159741 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:25.353884 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f7af1a59c77403fb32698dd243a17ef'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:25.365405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f7af1a59c77403fb32698dd243a17ef'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:25.366071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f7af1a59c77403fb32698dd243a17ef'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:25.372759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f7af1a59c77403fb32698dd243a17ef'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:25.436843 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f7af1a59c77403fb32698dd243a17ef'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:25.443635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1366/2733] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:25 2022] DELETE /identity/v3/users/3f7af1a59c77403fb32698dd243a17ef => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:25.448106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:25.448592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:25.448710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:25.467936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:25.469120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:25.526008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1368/2734] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:25.535672 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:25.537250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:25.593820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af68862-bbdc-4fbb-9aa5-947572b47dd2 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:25.597491 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:25.598084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:25.598365 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:25.598634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:25.608059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:25.609482 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:25.618816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1367/2735] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:26.061872 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:26.070877 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:26.077178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de8209-ea0f-4dc9-96dc-e971237782de None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:26.081974 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:26.082599 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:26.082875 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:26.083147 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] PATH_INFO: `/v3/projects/91304bd229c440aa95f52f2ae46f6a74` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:26.084447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] RBAC: Authorizing `identity:delete_project(project_id=91304bd229c440aa95f52f2ae46f6a74)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:26.086749 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:26.191442 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91304bd229c440aa95f52f2ae46f6a74', 'request_id': 'req-20ae3890-0190-400a-bc33-5a3e19ce129f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.196911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91304bd229c440aa95f52f2ae46f6a74', 'request_id': 'req-20ae3890-0190-400a-bc33-5a3e19ce129f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.202989 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91304bd229c440aa95f52f2ae46f6a74', 'request_id': 'req-20ae3890-0190-400a-bc33-5a3e19ce129f'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.293410 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] The token cache is being invalidate because project 91304bd229c440aa95f52f2ae46f6a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:26.293695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.294307 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.295822 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1369/2736] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:26 2022] DELETE /identity/v3/projects/91304bd229c440aa95f52f2ae46f6a74 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:26.299817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:26.310587 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585dd02-f7ee-408e-b62d-49ece3b90ef4 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:26.370997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:26.371931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:26.372306 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:26.372677 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] PATH_INFO: `/v3/users/4f67c6a5f1f744d7bcc02d9dd9bb7fbf` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:26.374586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f67c6a5f1f744d7bcc02d9dd9bb7fbf)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:26.379886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:26.379886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:26.598877 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f67c6a5f1f744d7bcc02d9dd9bb7fbf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.611532 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f67c6a5f1f744d7bcc02d9dd9bb7fbf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.612281 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f67c6a5f1f744d7bcc02d9dd9bb7fbf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.619167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f67c6a5f1f744d7bcc02d9dd9bb7fbf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.693257 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3d390ebd-45d4-4248-a4f0-03bef552fd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f67c6a5f1f744d7bcc02d9dd9bb7fbf'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:26.699652 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1368/2737] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:26 2022] DELETE /identity/v3/users/4f67c6a5f1f744d7bcc02d9dd9bb7fbf => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:27.118544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.120008 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:27.137479 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae3890-0190-400a-bc33-5a3e19ce129f admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.196200 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.196849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.197178 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.197463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] PATH_INFO: `/v3/projects/46a05029560a40f497793d95775512bd` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.198592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=46a05029560a40f497793d95775512bd)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:27.200522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:27.216541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:27.317904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46a05029560a40f497793d95775512bd', 'request_id': 'req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:27.323831 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46a05029560a40f497793d95775512bd', 'request_id': 'req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:27.329783 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46a05029560a40f497793d95775512bd', 'request_id': 'req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:27.408467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] The token cache is being invalidate because project 46a05029560a40f497793d95775512bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:27.408755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:27.409405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-9d6487ea-b5c6-4dd0-9ac5-86bf69bcdcaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:27.410898 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1370/2738] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:27 2022] DELETE /identity/v3/projects/46a05029560a40f497793d95775512bd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:27.417046 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.417628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.417628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.437446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:27.438644 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:27.498253 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1369/2739] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:27.506073 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.506399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.506778 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.526489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:27.527702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:27.535447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1371/2740] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:27.539600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.548925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a1577-5eb4-44b5-a016-8bdcc08f8ba1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.554094 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.554710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.554990 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.555246 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.555880 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:41:27.556174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:27.557729 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:27.569452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1370/2741] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:41:27.573539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.582670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a23776-edc8-47e4-b803-4bbaf82f0a22 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.587819 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.588404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.588678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.588948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.589568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:27.591487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:27.714234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1372/2742] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:27 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:41:27.719727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.730139 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62edc4e-a4ed-4166-a9e4-7ec1815d8c15 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.784320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.784925 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.785210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.785489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.786102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:27.788773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:27.799784 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:27.895120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1371/2743] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:27 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:27.899064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.909647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c8bcf-b1ab-4654-8333-212025fad875 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.915018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.915643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.915917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.916187 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.916823 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:27.918420 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:27.929364 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1373/2744] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:27.933146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.943125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a1b17-bcc0-4b3e-87f5-647de5a6cab1 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.947979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01951d5c-5367-4e4d-931a-e174173bb7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.948567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01951d5c-5367-4e4d-931a-e174173bb7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.948846 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01951d5c-5367-4e4d-931a-e174173bb7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.949116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01951d5c-5367-4e4d-931a-e174173bb7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.949777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01951d5c-5367-4e4d-931a-e174173bb7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:27.951374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01951d5c-5367-4e4d-931a-e174173bb7e4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:27.962787 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1372/2745] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:27.966779 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:27.977395 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a060138-b6f7-4b3e-8c89-20ae00225cf2 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:27.982025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:27.982664 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:27.982936 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:27.983295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] PATH_INFO: `/v3/projects/c4f561ad1b0541099b26f6bacda96514/users/700ebb85ca32486ab3ea29964b40ba03/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:27.998567 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4f561ad1b0541099b26f6bacda96514, user_id=700ebb85ca32486ab3ea29964b40ba03, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.001332 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:28.100914 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1374/2746] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:27 2022] PUT /identity/v3/projects/c4f561ad1b0541099b26f6bacda96514/users/700ebb85ca32486ab3ea29964b40ba03/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:28.164586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.164865 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.165138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.191836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] MFA Rules not processed for user `700ebb85ca32486ab3ea29964b40ba03`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:28.192980 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:28.263622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1373/2747] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:28.268150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:28.329519 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9494ee0-7cb6-45a7-a027-d1a7d0a4ed6a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:28.334310 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:28.335466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.335847 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.336120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.336732 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.338624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:28.426857 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1375/2748] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:28 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:28.431356 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:28.442320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f346e-b53b-4a20-a4a6-118f06801b59 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:28.498415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:28.499020 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.499331 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.499601 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.500213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.502104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:28.608662 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1374/2749] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:28 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:28.612603 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:28.623325 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586530b-a4c6-405b-b3a0-28d50a6310d6 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:28.628136 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:28.628730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.629038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.629317 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.629997 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.632341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:28.642876 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1376/2750] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:28.646803 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:28.657511 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c1d67-d4a2-43db-907a-aa99274fef65 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:28.662086 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2051b-8b62-4e9a-8add-2f5e636e678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:28.662718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2051b-8b62-4e9a-8add-2f5e636e678f admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.663021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2051b-8b62-4e9a-8add-2f5e636e678f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.663392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2051b-8b62-4e9a-8add-2f5e636e678f admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.664030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2051b-8b62-4e9a-8add-2f5e636e678f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.665594 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2051b-8b62-4e9a-8add-2f5e636e678f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:28.675526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1375/2751] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:28.679750 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:28.689681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc71961-2b84-4cb3-8069-6df567583586 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:28.694942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:28.695546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.695828 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.696106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] PATH_INFO: `/v3/projects/ae6b5e8210214288bed7c10969609be2/users/ad4c69740f2748f397baa220cc29ea61/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.706982 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:28.711775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6b5e8210214288bed7c10969609be2, user_id=ad4c69740f2748f397baa220cc29ea61, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.714415 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:28.821009 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1377/2752] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:28 2022] PUT /identity/v3/projects/ae6b5e8210214288bed7c10969609be2/users/ad4c69740f2748f397baa220cc29ea61/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:28.831480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.831765 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.832033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.860377 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] MFA Rules not processed for user `ad4c69740f2748f397baa220cc29ea61`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:28.861582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:28.902646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:28.924933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1376/2753] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:41:28.930006 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:28.991350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c97f9c-0b46-4de5-aad8-c07de65e12bc admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:28.995841 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:28.996432 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:28.996707 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:28.996975 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:28.997601 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:28.999522 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.095670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1378/2754] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:28 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:29.099576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:29.111542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443db07-6c00-4e51-b8fd-93a7c8af45cb None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:29.168567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:29.169179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.169458 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.169734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.170354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:29.172437 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.249157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1377/2755] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:29 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:29.253122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:29.263579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a05cb-432f-47e0-9907-214091920d70 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:29.268404 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:29.269049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.269406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.269678 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.270355 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:29.272182 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.283492 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1379/2756] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:29.291059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:29.301157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737477e7-fe6b-4fb4-8b80-f645ac61a7f8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:29.306489 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:29.307144 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.307446 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.307715 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] PATH_INFO: `/v3/projects/4089b9749ee44ad981ec7bd4f9f51215/users/6fa638e27a9f4f03b60ea96d31781947/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.322193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4089b9749ee44ad981ec7bd4f9f51215, user_id=6fa638e27a9f4f03b60ea96d31781947, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:29.324894 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.386089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1378/2757] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:29 2022] PUT /identity/v3/projects/4089b9749ee44ad981ec7bd4f9f51215/users/6fa638e27a9f4f03b60ea96d31781947/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:29.390011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:29.400839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5b7d3-9e42-4456-974c-2f287e968e75 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:29.460628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da8559c-f11a-4739-944d-962fe57ffb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:29.461459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8559c-f11a-4739-944d-962fe57ffb79 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.461744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8559c-f11a-4739-944d-962fe57ffb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.462018 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8559c-f11a-4739-944d-962fe57ffb79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.462662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8559c-f11a-4739-944d-962fe57ffb79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:29.464308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da8559c-f11a-4739-944d-962fe57ffb79 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.474323 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1380/2758] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:29.479167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:29.489634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ac54a-9dcd-45a4-944d-2aaf95be44cd admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:29.494902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:29.495590 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.495863 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.496132 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] PATH_INFO: `/v3/projects/4089b9749ee44ad981ec7bd4f9f51215/users/6fa638e27a9f4f03b60ea96d31781947/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.498025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4089b9749ee44ad981ec7bd4f9f51215, user_id=6fa638e27a9f4f03b60ea96d31781947, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:29.505423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.511311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1379/2759] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:29 2022] PUT /identity/v3/projects/4089b9749ee44ad981ec7bd4f9f51215/users/6fa638e27a9f4f03b60ea96d31781947/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:29.522685 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.522925 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.523137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.551467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] MFA Rules not processed for user `6fa638e27a9f4f03b60ea96d31781947`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:29.552904 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:29.601812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:29.625624 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1381/2760] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:29.641680 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:29.643543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:29.714028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fab615-fdf1-41cb-a64c-76f58e975354 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:29.717820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601e222-4313-4220-a143-87c8747c0f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:29.718422 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601e222-4313-4220-a143-87c8747c0f7f service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:29.718747 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601e222-4313-4220-a143-87c8747c0f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:29.719083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601e222-4313-4220-a143-87c8747c0f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:29.768825 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:29.778564 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601e222-4313-4220-a143-87c8747c0f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:29.782024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601e222-4313-4220-a143-87c8747c0f7f service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:29.788958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1380/2761] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:32.806313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:32.816362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71154e77-e8b2-465f-a132-243c9a8d9b32 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:32.820896 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:32.821407 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:32.821681 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:32.821947 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:32.881350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:32.882939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:32.895429 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1382/2762] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:32.910103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:32.910580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:32.910700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:32.930383 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] MFA Rules not processed for user `322afd3893ac4c2ba4ab4d449f4d9159`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:32.931657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:32.985217 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1381/2763] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:34.209912 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:34.218513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6394aab-26c2-40ea-8359-18090cceded1 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:34.222037 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e1209-65ca-4e90-affa-3f22c19b337d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:34.222625 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1209-65ca-4e90-affa-3f22c19b337d service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:34.222900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1209-65ca-4e90-affa-3f22c19b337d service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:34.223167 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e1209-65ca-4e90-affa-3f22c19b337d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:34.232612 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e1209-65ca-4e90-affa-3f22c19b337d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:34.234019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e1209-65ca-4e90-affa-3f22c19b337d service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:34.243301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1383/2764] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:38.876314 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:38.943308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38a8e-d7c4-420f-bb45-54a56d9b29c7 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:38.948441 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:38.949093 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:38.949560 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:38.949841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] PATH_INFO: `/v3/users/700ebb85ca32486ab3ea29964b40ba03` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:38.951073 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=700ebb85ca32486ab3ea29964b40ba03)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:38.952992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:39.120779 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '700ebb85ca32486ab3ea29964b40ba03'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:39.136344 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '700ebb85ca32486ab3ea29964b40ba03'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:39.137151 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '700ebb85ca32486ab3ea29964b40ba03'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:39.145425 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '700ebb85ca32486ab3ea29964b40ba03'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:39.236439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '700ebb85ca32486ab3ea29964b40ba03'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:39.243373 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1382/2765] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:38 2022] DELETE /identity/v3/users/700ebb85ca32486ab3ea29964b40ba03 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:39.247889 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:39.248386 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:39.248496 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:39.268211 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:39.269399 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:39.327022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1384/2766] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:39.337612 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:39.339005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:39.353745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:39.394704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c71956-59f1-40ed-8138-7753a69bb44b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:39.398837 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:39.399470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:39.399744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:39.400011 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:39.409116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:39.410528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:39.420540 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1383/2767] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:39.947961 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:39.949488 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:39.964123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb92a6b-7fdc-4515-b2ef-d5c98869fd32 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:39.969022 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:39.969741 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:39.970016 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:39.970840 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] PATH_INFO: `/v3/projects/c4f561ad1b0541099b26f6bacda96514` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:39.972234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4f561ad1b0541099b26f6bacda96514)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:39.974120 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:40.071873 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4f561ad1b0541099b26f6bacda96514', 'request_id': 'req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.077475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4f561ad1b0541099b26f6bacda96514', 'request_id': 'req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.083758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4f561ad1b0541099b26f6bacda96514', 'request_id': 'req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.086517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:40.156341 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] The token cache is being invalidate because project c4f561ad1b0541099b26f6bacda96514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:40.156633 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.157246 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.158716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1385/2768] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:39 2022] DELETE /identity/v3/projects/c4f561ad1b0541099b26f6bacda96514 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:40.162711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:40.172792 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6f71f-7bcc-4380-b7dc-f966c35feda6 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:40.229369 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:40.230025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:40.230310 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:40.230583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] PATH_INFO: `/v3/users/ad4c69740f2748f397baa220cc29ea61` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:40.231726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4c69740f2748f397baa220cc29ea61)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:40.233634 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:40.371392 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4c69740f2748f397baa220cc29ea61'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.384223 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4c69740f2748f397baa220cc29ea61'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.384917 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4c69740f2748f397baa220cc29ea61'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.392198 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4c69740f2748f397baa220cc29ea61'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.436975 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4c69740f2748f397baa220cc29ea61'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:40.443155 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1384/2769] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:40 2022] DELETE /identity/v3/users/ad4c69740f2748f397baa220cc29ea61 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:40.814544 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:40.816069 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:40.830221 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d5ee1c-38ed-4b34-b65b-989ab3417ca7 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:40.887878 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:40.888557 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:40.888835 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:40.889108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] PATH_INFO: `/v3/projects/ae6b5e8210214288bed7c10969609be2` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:40.890231 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6b5e8210214288bed7c10969609be2)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:40.892302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:41.002850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6b5e8210214288bed7c10969609be2', 'request_id': 'req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.008509 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6b5e8210214288bed7c10969609be2', 'request_id': 'req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.014716 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6b5e8210214288bed7c10969609be2', 'request_id': 'req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.099108 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] The token cache is being invalidate because project ae6b5e8210214288bed7c10969609be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:41.099523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.100226 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.102805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1386/2770] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:40 2022] DELETE /identity/v3/projects/ae6b5e8210214288bed7c10969609be2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:41.108395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:41.111507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:41.122932 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9176b-8892-4589-9023-33a7d7598fb8 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:41.155210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:41.178222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:41.178854 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:41.179130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:41.179596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] PATH_INFO: `/v3/users/6fa638e27a9f4f03b60ea96d31781947` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:41.180720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa638e27a9f4f03b60ea96d31781947)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:41.182623 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:41.347858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa638e27a9f4f03b60ea96d31781947'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.359754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa638e27a9f4f03b60ea96d31781947'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.360431 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa638e27a9f4f03b60ea96d31781947'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.367573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa638e27a9f4f03b60ea96d31781947'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.430498 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-87a6b578-9153-4826-bd78-e9b7ed936032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa638e27a9f4f03b60ea96d31781947'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:41.437284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1385/2771] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:41 2022] DELETE /identity/v3/users/6fa638e27a9f4f03b60ea96d31781947 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:41.895934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:41.897385 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:41.915527 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca4abf-c6ab-4d8b-bf83-1129d13ce463 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:41.972953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:41.973576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:41.973855 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:41.974126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] PATH_INFO: `/v3/projects/4089b9749ee44ad981ec7bd4f9f51215` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:41.975248 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4089b9749ee44ad981ec7bd4f9f51215)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:41.977690 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.072758 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:42.089271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4089b9749ee44ad981ec7bd4f9f51215', 'request_id': 'req-b2b0bd35-751e-404e-aee6-f1ed1e878722'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:42.094702 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4089b9749ee44ad981ec7bd4f9f51215', 'request_id': 'req-b2b0bd35-751e-404e-aee6-f1ed1e878722'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:42.101596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4089b9749ee44ad981ec7bd4f9f51215', 'request_id': 'req-b2b0bd35-751e-404e-aee6-f1ed1e878722'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:42.186644 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] The token cache is being invalidate because project 4089b9749ee44ad981ec7bd4f9f51215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:42.186887 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:42.187484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b2b0bd35-751e-404e-aee6-f1ed1e878722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:42.188967 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1387/2772] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:41 2022] DELETE /identity/v3/projects/4089b9749ee44ad981ec7bd4f9f51215 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:42.194913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.195627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.195627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.216148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:42.217439 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:42.280342 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1386/2773] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:42.288820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.289106 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.289470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.308733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:42.309933 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:42.317677 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1388/2774] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:42.322061 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.331580 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badae864-25cc-4899-b138-2c0bf578e2ff None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.336897 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.337480 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.337755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.338048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.338678 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:41:42.338978 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.340576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.352146 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1387/2775] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:41:42.356234 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.365540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286b4fc-4fef-4c28-9396-635d449f5e39 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.370330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.371104 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.371217 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.371465 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.372114 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.373984 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.449735 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1389/2776] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:41:42 2022] POST /identity/v3/projects => generated 369 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:42.453748 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.464575 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e12bb-5504-492f-b781-29bafe9d37ca admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.503731 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:42.519608 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.520204 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.520483 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.520756 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.521387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.524044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.608807 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1388/2777] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:41:42 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:41:42.612991 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.623444 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8558655-f51f-4973-8fb1-b352688f7ac7 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.628174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.628770 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.629053 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.629335 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.630057 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.632193 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.642712 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1390/2778] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 00:41:42.646889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.658418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efca883-d6bf-4d12-8bfb-fa43838576ab admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.663160 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536ca05f-7970-44a5-a207-bd5e9cbfaa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.663954 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ca05f-7970-44a5-a207-bd5e9cbfaa21 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.664282 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ca05f-7970-44a5-a207-bd5e9cbfaa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.664583 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536ca05f-7970-44a5-a207-bd5e9cbfaa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.665213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ca05f-7970-44a5-a207-bd5e9cbfaa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.666844 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536ca05f-7970-44a5-a207-bd5e9cbfaa21 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.678109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1389/2779] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:41:42.682517 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.692950 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb409528-e5f1-4a13-af42-9c0d5cff6b74 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.698153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.699927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.699927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.699927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] PATH_INFO: `/v3/projects/636b5b08dde64ebeb09291c0b62704c7/users/fe70d900ff2f443c973c31a817d67527/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.715596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=636b5b08dde64ebeb09291c0b62704c7, user_id=fe70d900ff2f443c973c31a817d67527, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.718240 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.780260 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1391/2780] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:41:42 2022] PUT /identity/v3/projects/636b5b08dde64ebeb09291c0b62704c7/users/fe70d900ff2f443c973c31a817d67527/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:41:42.799019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.799435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.799714 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.827103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] MFA Rules not processed for user `fe70d900ff2f443c973c31a817d67527`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:42.828332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:42.890543 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1390/2781] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:42.901759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:42.903463 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:42.962124 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37eee-9181-4229-ac84-9c0a54dc41fe admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:42.965771 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897af8e-1f25-4f37-ab92-145077eff316 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:42.966372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897af8e-1f25-4f37-ab92-145077eff316 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:42.966649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897af8e-1f25-4f37-ab92-145077eff316 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:42.966915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897af8e-1f25-4f37-ab92-145077eff316 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:42.975049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897af8e-1f25-4f37-ab92-145077eff316 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:42.977079 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897af8e-1f25-4f37-ab92-145077eff316 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:42.985759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1392/2782] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:43.014473 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:43.075345 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e846-09a9-42e5-96c7-1abcf9fa0670 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:43.079763 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:43.080371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:43.080646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:43.080916 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] PATH_INFO: `/v3/users/fe70d900ff2f443c973c31a817d67527` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:43.082019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe70d900ff2f443c973c31a817d67527)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:43.084023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:43.255556 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe70d900ff2f443c973c31a817d67527'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:43.260015 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:43.267962 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe70d900ff2f443c973c31a817d67527'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:43.268660 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe70d900ff2f443c973c31a817d67527'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:43.276080 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe70d900ff2f443c973c31a817d67527'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:43.345414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe70d900ff2f443c973c31a817d67527'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:43.354116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1391/2783] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:41:43 2022] DELETE /identity/v3/users/fe70d900ff2f443c973c31a817d67527 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:43.358552 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:43.358860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:43.359135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:43.377915 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:41:43.379122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:43.429036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:43.441466 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1393/2784] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:41:43.451554 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:43.453407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:43.512244 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245278a-bdc6-4acc-a71d-ec4804a01726 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:43.515751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:43.516350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:43.516664 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:43.516937 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:43.526646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:43.528179 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:43.537635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1392/2785] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:41:44.107858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:41:44.109377 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:41:44.122290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c6be3-8533-466f-8970-8d7aa3368307 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:41:44.127472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:41:44.128110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:41:44.128389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:41:44.128661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] PATH_INFO: `/v3/projects/636b5b08dde64ebeb09291c0b62704c7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:41:44.129792 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=636b5b08dde64ebeb09291c0b62704c7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:41:44.131795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:41:44.249396 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636b5b08dde64ebeb09291c0b62704c7', 'request_id': 'req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:44.255007 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636b5b08dde64ebeb09291c0b62704c7', 'request_id': 'req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:44.261418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636b5b08dde64ebeb09291c0b62704c7', 'request_id': 'req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:44.338000 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] The token cache is being invalidate because project 636b5b08dde64ebeb09291c0b62704c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:41:44.338309 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:44.338917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:41:44.340528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1394/2786] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:41:44 2022] DELETE /identity/v3/projects/636b5b08dde64ebeb09291c0b62704c7 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:41:53.533481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=3,keystone:SELECT=36,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:41:54.334647 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:42:25.129671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:42:25.131084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:42:25.187256 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac911102-7e2f-4dec-9135-2d33e3991e8d service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:42:25.235755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:42:25.236397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:42:25.236720 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:42:25.237023 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:42:25.238808 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:42:25.308048 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:42:25.309476 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:42:25.320956 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1393/2787] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:42:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:42:28.180362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:42:28.181767 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:42:28.238467 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578462cc-2aa8-4a7a-b5e5-4fab3e775b91 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:42:28.242550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:42:28.243150 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:42:28.243457 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:42:28.243731 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:42:28.245450 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:42:28.262881 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:42:28.264354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740a89b-3e1f-4675-b795-df9143b6db73 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:42:28.276072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1395/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:42:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:42:35.315565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=74 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:42:38.270525 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=26 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:43:17.224171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:17.225866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:17.241156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d02b-f5dc-4859-8368-1a42ad66c53d service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:17.247082 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:17.247710 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:17.247988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:17.248259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:17.249979 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:17.264941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:17.266545 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:17.278889 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1394/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:22.829274 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:22.829756 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:22.830005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:22.849586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] MFA Rules not processed for user `1f22b554ae524ccf9a8777ac24cdd688`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:43:22.850775 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:22.904569 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1396/2790] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:43:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:43:23.205096 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:23.206429 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:23.218112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a70d9-768d-46ef-b948-ddb2ebf3cb10 service nova] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:23.221686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:23.222273 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:23.222551 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:23.222824 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:23.224494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:23.233734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:23.235142 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:23.243494 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1395/2791] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:23.622112 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:23.623540 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:23.698662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329375a2-c2ea-460c-9ffd-e306382f6064 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:23.703670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:23.704302 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:23.704580 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:23.704852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:23.706566 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:23.721329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:23.721329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:23.730313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1397/2792] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:26.498841 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:26.507694 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d94e63-409c-42db-ad3d-36f170e863d3 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:26.511790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:26.512374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:26.512651 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:26.512922 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:26.514643 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:26.525986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:26.526859 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:26.535406 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1396/2793] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:31.502948 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:31.504523 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:31.521703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e070dd-2df9-486d-ad4a-d0f6d60fb160 service placement] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:31.527373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1031699d7da4a76965035faa9305dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:31.527986 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:31.528263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:31.528531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:31.530311 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:31.543097 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:31.544821 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:31.558706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1398/2794] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:36.532235 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=18 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:43:41.551045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=63 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:43:45.889810 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:45.900084 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b4c4be-1649-4fcc-86bc-e8b68d4bd2ef service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:45.903931 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:45.904534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:45.904817 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:45.905213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:45.906958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:45.972477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:45.973957 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:45.984586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1397/2795] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:49.377942 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:49.379892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:49.394485 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5f52b-07f8-48e4-a92e-d1893d181fee service nova] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:49.400197 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:49.400861 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:49.401144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:49.401475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:49.403242 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:49.416229 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:49.419700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:49.428676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1399/2796] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:43:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:43:53.160759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:53.162762 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:53.178943 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ee8aa-d7b5-4f41-93d4-3bee8c5a66e6 service neutron] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:53.184097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:53.184718 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:53.184992 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:53.185260 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] PATH_INFO: `/v3/users/322afd3893ac4c2ba4ab4d449f4d9159` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:53.186353 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=322afd3893ac4c2ba4ab4d449f4d9159)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:53.188271 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:53.344481 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '322afd3893ac4c2ba4ab4d449f4d9159'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.356396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '322afd3893ac4c2ba4ab4d449f4d9159'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.357054 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '322afd3893ac4c2ba4ab4d449f4d9159'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.363661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '322afd3893ac4c2ba4ab4d449f4d9159'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.416988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '322afd3893ac4c2ba4ab4d449f4d9159'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.422826 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1398/2797] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:43:53 2022] DELETE /identity/v3/users/322afd3893ac4c2ba4ab4d449f4d9159 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:43:53.659126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:53.673023 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4f3f4-7925-4b7f-894a-3c05a7f30ed2 service placement] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:53.733313 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:53.733939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:53.734215 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:53.734489 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] PATH_INFO: `/v3/projects/a22da44baa6b472e91d99fbcb03b583c` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:53.736081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22da44baa6b472e91d99fbcb03b583c)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:53.738010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:53.846135 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22da44baa6b472e91d99fbcb03b583c', 'request_id': 'req-bbb41658-891e-48e6-b27a-b69e5ca15102'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.851641 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22da44baa6b472e91d99fbcb03b583c', 'request_id': 'req-bbb41658-891e-48e6-b27a-b69e5ca15102'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.857718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22da44baa6b472e91d99fbcb03b583c', 'request_id': 'req-bbb41658-891e-48e6-b27a-b69e5ca15102'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.931502 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] The token cache is being invalidate because project a22da44baa6b472e91d99fbcb03b583c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:43:53.931725 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.932351 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:53.933879 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1400/2798] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:43:53 2022] DELETE /identity/v3/projects/a22da44baa6b472e91d99fbcb03b583c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:43:53.937968 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:53.942236 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:53.952982 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2673e1-d9b0-46bf-b84a-03a5202e0cb4 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:54.008611 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:54.009229 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:54.009507 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:54.009778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] PATH_INFO: `/v3/users/a818a59ba73c455c9d11092655901cd9` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:54.010910 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a818a59ba73c455c9d11092655901cd9)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:54.012852 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:54.018072 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:43:54.180119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a818a59ba73c455c9d11092655901cd9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.191948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a818a59ba73c455c9d11092655901cd9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.192546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a818a59ba73c455c9d11092655901cd9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.199742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a818a59ba73c455c9d11092655901cd9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.250858 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a818a59ba73c455c9d11092655901cd9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.256969 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1399/2799] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:43:53 2022] DELETE /identity/v3/users/a818a59ba73c455c9d11092655901cd9 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:43:54.511071 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:43:54.512553 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:43:54.525642 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb41658-891e-48e6-b27a-b69e5ca15102 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:43:54.587424 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:43:54.588059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:43:54.588337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:43:54.588609 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] PATH_INFO: `/v3/projects/ad2ab7aaac7341739eee2f43ea9f161f` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:43:54.589730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2ab7aaac7341739eee2f43ea9f161f)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:43:54.591675 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:43:54.614596 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:43:54.686957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2ab7aaac7341739eee2f43ea9f161f', 'request_id': 'req-ede471f4-1e2b-4a0a-94c1-530439fd972d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.692470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2ab7aaac7341739eee2f43ea9f161f', 'request_id': 'req-ede471f4-1e2b-4a0a-94c1-530439fd972d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.699490 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2ab7aaac7341739eee2f43ea9f161f', 'request_id': 'req-ede471f4-1e2b-4a0a-94c1-530439fd972d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.770968 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] The token cache is being invalidate because project ad2ab7aaac7341739eee2f43ea9f161f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:43:54.771166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.771815 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:43:54.773300 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1401/2800] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:43:54 2022] DELETE /identity/v3/projects/ad2ab7aaac7341739eee2f43ea9f161f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:44:04.253391 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:04.846171 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:54.184024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:54.185742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:54.201407 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ff5b4-2c51-434a-b567-e42abaf4cf3d admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:54.258109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:54.258855 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:54.259129 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:54.259433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] PATH_INFO: `/v3/users/1f22b554ae524ccf9a8777ac24cdd688` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:54.260548 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f22b554ae524ccf9a8777ac24cdd688)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:54.262493 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:54.481463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f22b554ae524ccf9a8777ac24cdd688'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:54.494044 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f22b554ae524ccf9a8777ac24cdd688'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:54.494726 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f22b554ae524ccf9a8777ac24cdd688'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:54.501544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f22b554ae524ccf9a8777ac24cdd688'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:54.582088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-6377ec5a-c9c7-4923-b86f-73caa35c7563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f22b554ae524ccf9a8777ac24cdd688'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:54.588461 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1400/2801] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:44:54 2022] DELETE /identity/v3/users/1f22b554ae524ccf9a8777ac24cdd688 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:44:54.859586 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:54.863739 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:54.875676 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede471f4-1e2b-4a0a-94c1-530439fd972d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:54.947209 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:54.951837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:54.951837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:54.951837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] PATH_INFO: `/v3/projects/2c27d244bd514766b955d1a4aba239eb` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:54.951837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c27d244bd514766b955d1a4aba239eb)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:54.951837 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.069611 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c27d244bd514766b955d1a4aba239eb', 'request_id': 'req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:55.075058 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c27d244bd514766b955d1a4aba239eb', 'request_id': 'req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:55.081383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c27d244bd514766b955d1a4aba239eb', 'request_id': 'req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:55.158452 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] The token cache is being invalidate because project 2c27d244bd514766b955d1a4aba239eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91987) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:44:55.158670 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:55.159321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d87c7ea0-8c87-4cd9-97e4-8dd88b2262d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:44:55.160854 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1402/2802] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:44:54 2022] DELETE /identity/v3/projects/2c27d244bd514766b955d1a4aba239eb => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:44:55.167354 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.167712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.167934 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.187758 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:44:55.188947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:55.246737 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1401/2803] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:44:55.254604 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.254891 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.255166 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.275244 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:44:55.276447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:55.284290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1403/2804] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:44:55.288362 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.298719 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122493e-1b4e-465e-a28e-16f1e65825dc None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:55.303395 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:55.303985 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.304264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.304573 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] PATH_INFO: `/v3/domains` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.305220 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91988) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:44:55.305518 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:55.307092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.318267 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1402/2805] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:44:55.322524 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.331753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86628a-bf11-47eb-82b7-0f7eee7fbf59 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:55.337384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:55.337970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.338247 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.338649 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.339312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:55.341204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.475090 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1404/2806] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:44:55 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:44:55.479716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.489716 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f6a27-a300-4e23-accb-f6ca99b27b88 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:55.493313 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:55.545759 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:55.546358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.546636 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.546907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.547613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:55.549515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.639838 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1403/2807] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:44:55 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:44:55.644139 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.655478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa790072-7da7-4fbb-ab9c-d498ef1d1309 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:55.660170 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:55.660848 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.661144 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.661397 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.662036 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:55.663695 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.673911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1405/2808] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:44:55.677888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.688528 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86c37-0a3b-44e2-a9f0-b0e40e79e048 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:55.693165 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3107ad-c670-4fd6-ab25-d4ced1ed7fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:55.693782 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3107ad-c670-4fd6-ab25-d4ced1ed7fad admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.694058 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3107ad-c670-4fd6-ab25-d4ced1ed7fad admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.694329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3107ad-c670-4fd6-ab25-d4ced1ed7fad admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.695504 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3107ad-c670-4fd6-ab25-d4ced1ed7fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:55.697138 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3107ad-c670-4fd6-ab25-d4ced1ed7fad admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.707948 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1404/2809] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:44:55.713065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.723288 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2a6e-79ea-473e-ac23-fdc563100d7b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:55.728558 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:55.729138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.729454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.729730 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] PATH_INFO: `/v3/projects/8801a1265630487596cf78081115435b/users/6bbaf48478434ffda7d9fb5224190922/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.745204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8801a1265630487596cf78081115435b, user_id=6bbaf48478434ffda7d9fb5224190922, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:55.747998 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:55.839843 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1406/2810] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:44:55 2022] PUT /identity/v3/projects/8801a1265630487596cf78081115435b/users/6bbaf48478434ffda7d9fb5224190922/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:44:55.852387 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:55.852688 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:55.853027 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:55.880285 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] MFA Rules not processed for user `6bbaf48478434ffda7d9fb5224190922`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:44:55.881477 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:55.944816 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1405/2811] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:44:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:44:55.949350 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:55.969026 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:56.010196 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dca79-00be-47f5-b545-9bd753ef5c4b admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.014737 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.015363 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.015643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.015913 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.016542 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.018436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.085864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1407/2812] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:44:55 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:44:56.090250 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.101050 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b06b1-2b4d-4068-bcbc-617da61426f2 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.156827 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.157419 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.157691 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.157958 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.158571 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.160488 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.210052 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1406/2813] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:44:56 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:44:56.214436 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.224870 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ceaee4-7fb7-454d-84d8-3b4829348304 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.229632 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.230816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.230816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.230816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.231426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.233039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.244383 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1408/2814] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:44:56.248402 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.254753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:56.259435 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2def8-71bb-4bdc-85ce-101615e41491 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.264113 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ae0163-1676-47c6-8813-76377e056935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.264645 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0163-1676-47c6-8813-76377e056935 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.264920 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0163-1676-47c6-8813-76377e056935 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.265188 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ae0163-1676-47c6-8813-76377e056935 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.265821 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae0163-1676-47c6-8813-76377e056935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.267444 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ae0163-1676-47c6-8813-76377e056935 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.277380 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1407/2815] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:44:56.281472 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.291636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2509af8-51b4-4b23-bb94-5065c3b234ff admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.296628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.302130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.302130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.302130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] PATH_INFO: `/v3/projects/7391fc29ebd444c6bad9682c2626a5da/users/7ba2028d58cf4199a3b2099041239cc7/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.313074 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7391fc29ebd444c6bad9682c2626a5da, user_id=7ba2028d58cf4199a3b2099041239cc7, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.315794 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.411087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1409/2816] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:44:56 2022] PUT /identity/v3/projects/7391fc29ebd444c6bad9682c2626a5da/users/7ba2028d58cf4199a3b2099041239cc7/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:44:56.421986 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.422275 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.422552 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.450812 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] MFA Rules not processed for user `7ba2028d58cf4199a3b2099041239cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:44:56.452109 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:56.523704 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1408/2817] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:44:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:44:56.528297 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.591081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006258d1-1c71-4c12-b16e-efe0ecab2b95 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.595939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.596554 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.596836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.597109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.597748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.599710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.698175 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1410/2818] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:44:56 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:44:56.702071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.713635 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed20f540-abf4-4a40-9256-8f43a8837f5e None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.768350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.768952 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.769270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.769568 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] PATH_INFO: `/v3/users` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.770185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.772119 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.862156 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1409/2819] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:44:56 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:44:56.866897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.877153 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89930fa-a200-48fc-b0b3-3454c7167d5d admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.881824 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.882550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.882733 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.883010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.883743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.885329 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:56.895897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1411/2820] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:44:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:44:56.900866 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:56.910576 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5447ce-b98f-4257-9a3e-e85538328433 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:56.915853 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:56.916486 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:56.916760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:56.917030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] PATH_INFO: `/v3/projects/ad93148f889f4330a2d074a9525ffd3f/users/06c5b7fe6fc14a23b33b4df0bef71a1d/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:56.931242 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad93148f889f4330a2d074a9525ffd3f, user_id=06c5b7fe6fc14a23b33b4df0bef71a1d, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:56.933882 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:57.016563 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1410/2821] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:44:56 2022] PUT /identity/v3/projects/ad93148f889f4330a2d074a9525ffd3f/users/06c5b7fe6fc14a23b33b4df0bef71a1d/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:44:57.020518 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:57.031455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bf788-00dc-4646-848e-f4eb2d2d1c54 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:57.070900 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:57.087744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b9be0-35f1-42cd-80e6-79f3fd16afe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:57.088337 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b9be0-35f1-42cd-80e6-79f3fd16afe3 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:57.088615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b9be0-35f1-42cd-80e6-79f3fd16afe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:57.088883 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b9be0-35f1-42cd-80e6-79f3fd16afe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:57.089539 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b9be0-35f1-42cd-80e6-79f3fd16afe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:57.091127 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b9be0-35f1-42cd-80e6-79f3fd16afe3 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:57.100533 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1412/2822] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:44:57.104751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:57.119438 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff333cd-886d-4e9a-a77b-ec4310475adb admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:57.119622 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:57.120186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:57.120471 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:57.120742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] PATH_INFO: `/v3/projects/ad93148f889f4330a2d074a9525ffd3f/users/06c5b7fe6fc14a23b33b4df0bef71a1d/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:57.122638 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad93148f889f4330a2d074a9525ffd3f, user_id=06c5b7fe6fc14a23b33b4df0bef71a1d, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:57.125269 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:57.136028 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1411/2823] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:44:57 2022] PUT /identity/v3/projects/ad93148f889f4330a2d074a9525ffd3f/users/06c5b7fe6fc14a23b33b4df0bef71a1d/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:44:57.146576 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:57.146856 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:57.147126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:57.174308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] MFA Rules not processed for user `06c5b7fe6fc14a23b33b4df0bef71a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:44:57.175521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:57.241405 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1413/2824] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:44:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:44:57.252102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:57.253751 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:57.294433 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:44:57.311033 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec00915-828f-4bbf-9023-dfb54717902e admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:57.314697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:57.315320 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:57.315602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:57.315870 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:57.325283 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:57.326700 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:57.336535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1412/2825] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:44:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:44:58.352091 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:44:58.353510 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:44:58.365673 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e0e33-927c-47e9-803d-3d4528de22a6 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:44:58.369451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:44:58.370039 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:44:58.370314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:44:58.370579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:44:58.426362 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:44:58.427820 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:44:58.436671 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1414/2826] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:44:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:03.426596 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:03.436185 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e39ae8-db3d-4e90-b959-770ab9e44cba service cinder] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:03.439946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962c411-4c34-4955-96f4-bddc135ab95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:03.440526 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962c411-4c34-4955-96f4-bddc135ab95a service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:03.440798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962c411-4c34-4955-96f4-bddc135ab95a service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:03.441064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962c411-4c34-4955-96f4-bddc135ab95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:03.497742 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962c411-4c34-4955-96f4-bddc135ab95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:03.499166 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962c411-4c34-4955-96f4-bddc135ab95a service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:03.508211 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1413/2827] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:07.541225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:07.603401 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b60a6-870f-48fd-94df-1a9ffc00d1eb service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:07.607832 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:07.608474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:07.608746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:07.609017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] PATH_INFO: `/v3/users/6bbaf48478434ffda7d9fb5224190922` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:07.610117 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbaf48478434ffda7d9fb5224190922)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:07.612087 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:07.614376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=100 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:07.761592 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbaf48478434ffda7d9fb5224190922'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:07.774130 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbaf48478434ffda7d9fb5224190922'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:07.774795 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbaf48478434ffda7d9fb5224190922'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:07.781528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbaf48478434ffda7d9fb5224190922'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:07.833123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbaf48478434ffda7d9fb5224190922'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:07.840384 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1415/2828] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:07 2022] DELETE /identity/v3/users/6bbaf48478434ffda7d9fb5224190922 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:07.845066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:07.845586 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:07.845692 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:07.864222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:07.865426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:07.923582 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1414/2829] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:07.934131 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:07.935616 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:07.992357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b57de8-d9f4-49bb-9b32-355a29b5e9c5 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:07.996286 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:07.996884 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:07.997159 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:07.997426 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:08.006380 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:08.008969 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:08.018462 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1416/2830] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:08.429994 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:08.431467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:08.443945 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731c1e0-7df6-4446-9e7f-db4e210f1261 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:08.449455 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:08.450169 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:08.450632 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:08.450912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] PATH_INFO: `/v3/projects/8801a1265630487596cf78081115435b` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:08.452104 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8801a1265630487596cf78081115435b)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:08.454071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:08.541959 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8801a1265630487596cf78081115435b', 'request_id': 'req-75807376-d628-4496-bda7-56c8fc61032b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.547534 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8801a1265630487596cf78081115435b', 'request_id': 'req-75807376-d628-4496-bda7-56c8fc61032b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.555358 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8801a1265630487596cf78081115435b', 'request_id': 'req-75807376-d628-4496-bda7-56c8fc61032b'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.600570 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:08.605210 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] The token cache is being invalidate because project 8801a1265630487596cf78081115435b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:08.607902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.607902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.607902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1415/2831] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:08 2022] DELETE /identity/v3/projects/8801a1265630487596cf78081115435b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:08.612122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:08.622615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ddf37-1621-427b-a603-f4ab889e407b service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:08.678963 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:08.682531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:08.682531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:08.682531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] PATH_INFO: `/v3/users/7ba2028d58cf4199a3b2099041239cc7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:08.682531 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba2028d58cf4199a3b2099041239cc7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:08.683836 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:08.771646 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:08.875391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba2028d58cf4199a3b2099041239cc7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.887477 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba2028d58cf4199a3b2099041239cc7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.888143 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba2028d58cf4199a3b2099041239cc7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.894780 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba2028d58cf4199a3b2099041239cc7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.987359 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba2028d58cf4199a3b2099041239cc7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:08.994172 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1417/2832] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:08 2022] DELETE /identity/v3/users/7ba2028d58cf4199a3b2099041239cc7 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:45:09.383550 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:09.385021 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:09.400350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75807376-d628-4496-bda7-56c8fc61032b admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:09.458378 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:09.459004 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:09.459323 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:09.459599 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] PATH_INFO: `/v3/projects/7391fc29ebd444c6bad9682c2626a5da` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:09.461515 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7391fc29ebd444c6bad9682c2626a5da)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:09.463711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:09.577006 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7391fc29ebd444c6bad9682c2626a5da', 'request_id': 'req-9ee6d35e-3291-4421-b65c-2c558e2f5a45'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.583405 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7391fc29ebd444c6bad9682c2626a5da', 'request_id': 'req-9ee6d35e-3291-4421-b65c-2c558e2f5a45'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.590413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7391fc29ebd444c6bad9682c2626a5da', 'request_id': 'req-9ee6d35e-3291-4421-b65c-2c558e2f5a45'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.673170 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] The token cache is being invalidate because project 7391fc29ebd444c6bad9682c2626a5da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:09.673478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.674076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.675624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1416/2833] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:09 2022] DELETE /identity/v3/projects/7391fc29ebd444c6bad9682c2626a5da => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:09.685550 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:09.686977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:09.700591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6f049-7047-4767-b846-5d5a0d3f4d04 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:09.756742 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:09.758048 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:09.758322 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:09.758591 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] PATH_INFO: `/v3/users/06c5b7fe6fc14a23b33b4df0bef71a1d` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:09.759728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c5b7fe6fc14a23b33b4df0bef71a1d)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:09.761626 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:09.962190 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c5b7fe6fc14a23b33b4df0bef71a1d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.974068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c5b7fe6fc14a23b33b4df0bef71a1d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.974745 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c5b7fe6fc14a23b33b4df0bef71a1d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:09.982617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c5b7fe6fc14a23b33b4df0bef71a1d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.067555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-d17fc867-c2ab-47d1-8fb3-41338960e8b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c5b7fe6fc14a23b33b4df0bef71a1d'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.073814 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1418/2834] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:09 2022] DELETE /identity/v3/users/06c5b7fe6fc14a23b33b4df0bef71a1d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:10.491893 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:10.493669 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:10.506777 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d35e-3291-4421-b65c-2c558e2f5a45 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:10.564836 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:10.565470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:10.565754 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:10.566030 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] PATH_INFO: `/v3/projects/ad93148f889f4330a2d074a9525ffd3f` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:10.567143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad93148f889f4330a2d074a9525ffd3f)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:10.570675 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:10.690850 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:10.703332 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad93148f889f4330a2d074a9525ffd3f', 'request_id': 'req-3e5c729b-1cf6-4abc-93d6-76504095090c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.709125 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad93148f889f4330a2d074a9525ffd3f', 'request_id': 'req-3e5c729b-1cf6-4abc-93d6-76504095090c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.716066 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad93148f889f4330a2d074a9525ffd3f', 'request_id': 'req-3e5c729b-1cf6-4abc-93d6-76504095090c'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.816089 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] The token cache is being invalidate because project ad93148f889f4330a2d074a9525ffd3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:10.816289 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.816923 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-3e5c729b-1cf6-4abc-93d6-76504095090c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:10.818399 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1417/2835] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:10 2022] DELETE /identity/v3/projects/ad93148f889f4330a2d074a9525ffd3f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:10.825052 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:10.825546 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:10.825669 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:10.845709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:10.846911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:10.864723 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:10.906803 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1419/2836] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:10.915430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:10.915717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:10.915991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:10.935946 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:10.937147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:10.945815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1418/2837] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:10.950162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:10.959964 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023defff-249a-4b9c-b319-d9af2a17fa27 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:10.964858 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:10.965449 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:10.965727 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:10.965994 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:10.966630 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:45:10.966928 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:10.968553 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:10.980157 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1420/2838] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:45:10.984301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:10.994389 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede64983-98ed-44dc-b5f2-6df19fb14d11 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:10.999122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:10.999750 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.000031 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.000311 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.000944 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.002831 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.092233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1419/2839] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:45:10 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:45:11.097027 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.107269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979c5b8-1991-48f3-879c-7391fe7e4883 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.162628 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.163284 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.163755 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.164236 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.165459 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.167418 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.221414 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1421/2840] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:45:11 2022] POST /identity/v3/users => generated 305 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:11.225549 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.236158 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d9f27-15da-4a97-88ad-586f782ff8f5 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.240760 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.241351 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.242196 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.242628 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.243303 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.244890 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.254523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1420/2841] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:11.259308 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.269082 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef84c9c-7401-409f-862e-c4ae69519cab admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.274369 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc988d51-0485-4738-8b4f-2589ad54fd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.275110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc988d51-0485-4738-8b4f-2589ad54fd06 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.275447 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc988d51-0485-4738-8b4f-2589ad54fd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.275743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc988d51-0485-4738-8b4f-2589ad54fd06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.276373 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc988d51-0485-4738-8b4f-2589ad54fd06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.277949 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc988d51-0485-4738-8b4f-2589ad54fd06 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.289528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1422/2842] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:11.293661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.303736 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c18c5-8e96-431e-84cb-27f38d05d87a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.309111 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.309778 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.310057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.310329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] PATH_INFO: `/v3/projects/8218c123052a4a72b928ec03f4b189cc/users/646a6c089db5475caf4cb58216e02dd1/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.324520 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8218c123052a4a72b928ec03f4b189cc, user_id=646a6c089db5475caf4cb58216e02dd1, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.327147 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.410654 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1421/2843] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:45:11 2022] PUT /identity/v3/projects/8218c123052a4a72b928ec03f4b189cc/users/646a6c089db5475caf4cb58216e02dd1/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:11.422388 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.422744 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.423083 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.450753 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] MFA Rules not processed for user `646a6c089db5475caf4cb58216e02dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:11.451995 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:11.516225 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1423/2844] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:11.520671 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.580578 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a33a3-b459-4d7c-9c42-4dc76e71685a admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.585203 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.585798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.586078 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.586350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.587264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.589233 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.669814 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1422/2845] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:45:11 2022] POST /identity/v3/projects => generated 409 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:11.674109 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.685122 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e501-fca7-42d0-842e-fcce9e4e0a4d None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.697065 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:11.740538 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.741142 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.741443 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.741748 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.742357 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.744615 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.867766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1424/2846] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:45:11 2022] POST /identity/v3/users => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:11.872116 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.882627 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c136a-f43f-429c-b495-010eb4252fa7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.889941 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.890641 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.894105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.894105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.894105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.894105 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:11.905069 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1423/2847] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:11.909298 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:11.920910 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10a155-f6c2-45a1-b49c-1f4d4e2ac815 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:11.925743 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:11.926668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:11.927002 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:11.927301 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] PATH_INFO: `/v3/projects/6a436add251c44df82e43cb03d1b00bb/users/cad7f3bcb9254cdbb82d6ab242d7bb42/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:11.943162 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a436add251c44df82e43cb03d1b00bb, user_id=cad7f3bcb9254cdbb82d6ab242d7bb42, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:11.945850 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:12.039454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1425/2848] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:45:11 2022] PUT /identity/v3/projects/6a436add251c44df82e43cb03d1b00bb/users/cad7f3bcb9254cdbb82d6ab242d7bb42/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:12.043440 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:12.053357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f2724-d294-4d70-9300-0b4dd1b01363 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:12.081682 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:12.108713 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d12db5-a8f3-458b-ae08-b5d472cf0296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:12.109308 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d12db5-a8f3-458b-ae08-b5d472cf0296 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:12.109588 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d12db5-a8f3-458b-ae08-b5d472cf0296 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:12.109860 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d12db5-a8f3-458b-ae08-b5d472cf0296 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:12.110485 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d12db5-a8f3-458b-ae08-b5d472cf0296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:12.112112 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d12db5-a8f3-458b-ae08-b5d472cf0296 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:12.121600 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1424/2849] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:12.125784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:12.136687 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b71484-ce81-4b4b-83ed-f32bffde4729 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:12.141833 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:12.142164 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] REQUEST_METHOD: `PUT` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:12.142441 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:12.142709 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] PATH_INFO: `/v3/projects/6a436add251c44df82e43cb03d1b00bb/users/cad7f3bcb9254cdbb82d6ab242d7bb42/roles/ad68083fe37d48cba8726cfd30e43fb0` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:12.144638 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a436add251c44df82e43cb03d1b00bb, user_id=cad7f3bcb9254cdbb82d6ab242d7bb42, role_id=ad68083fe37d48cba8726cfd30e43fb0)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:12.147382 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:12.158693 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1426/2850] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:45:12 2022] PUT /identity/v3/projects/6a436add251c44df82e43cb03d1b00bb/users/cad7f3bcb9254cdbb82d6ab242d7bb42/roles/ad68083fe37d48cba8726cfd30e43fb0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:12.170421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:12.170709 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:12.171037 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:12.200466 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] MFA Rules not processed for user `cad7f3bcb9254cdbb82d6ab242d7bb42`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:12.202349 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:12.272222 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1425/2851] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:12 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:12.283763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:12.285487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:12.341897 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bb886-1973-4da3-93c1-ae6c3c645965 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:12.345475 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:12.346080 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:12.346354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:12.346623 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:12.355713 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:12.357134 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:12.366446 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1427/2852] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:12 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:12.694867 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:12.705174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71628702-72ca-4334-91dc-6beaeb12e422 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:12.708745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d3f6d-e5cb-4ec7-bd66-da2d1fccd0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:12.709335 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d3f6d-e5cb-4ec7-bd66-da2d1fccd0be service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:12.709615 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d3f6d-e5cb-4ec7-bd66-da2d1fccd0be service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:12.709887 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d3f6d-e5cb-4ec7-bd66-da2d1fccd0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:12.766602 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d3f6d-e5cb-4ec7-bd66-da2d1fccd0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:12.768053 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d3f6d-e5cb-4ec7-bd66-da2d1fccd0be service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:12.777064 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1426/2853] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:12 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:22.362338 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:22.563137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:22.624487 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee26b5-ff14-4e6d-b51b-097075064438 service cinder] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:22.629291 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:22.629911 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:22.630251 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:22.630619 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] PATH_INFO: `/v3/users/646a6c089db5475caf4cb58216e02dd1` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:22.631772 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=646a6c089db5475caf4cb58216e02dd1)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:22.633692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:22.772755 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=77 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:22.812885 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646a6c089db5475caf4cb58216e02dd1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:22.824662 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646a6c089db5475caf4cb58216e02dd1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:22.825314 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646a6c089db5475caf4cb58216e02dd1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:22.832011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646a6c089db5475caf4cb58216e02dd1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:22.913352 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646a6c089db5475caf4cb58216e02dd1'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:22.920700 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1428/2854] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:22 2022] DELETE /identity/v3/users/646a6c089db5475caf4cb58216e02dd1 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:22.925212 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:22.929542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:22.929542 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:22.946001 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:22.947200 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:23.004661 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1427/2855] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:23.014354 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:23.015866 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:23.072672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc8fe4-bc23-4ea4-abc2-422b519d0b5c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:23.076271 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:23.076865 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:23.077138 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:23.077406 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:23.086204 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:23.088692 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:23.097970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1429/2856] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:23.486804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:23.491478 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:23.501057 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eda919-96e6-437f-96d6-c1e860947179 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:23.506174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:23.506794 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:23.507070 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:23.507382 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] PATH_INFO: `/v3/projects/8218c123052a4a72b928ec03f4b189cc` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:23.508509 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8218c123052a4a72b928ec03f4b189cc)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:23.511120 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:23.611208 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8218c123052a4a72b928ec03f4b189cc', 'request_id': 'req-7f5e7656-8a86-4633-be55-eb3c661c61ba'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.616776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8218c123052a4a72b928ec03f4b189cc', 'request_id': 'req-7f5e7656-8a86-4633-be55-eb3c661c61ba'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.623780 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8218c123052a4a72b928ec03f4b189cc', 'request_id': 'req-7f5e7656-8a86-4633-be55-eb3c661c61ba'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.713861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] The token cache is being invalidate because project 8218c123052a4a72b928ec03f4b189cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:23.714157 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.714783 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.716329 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1428/2857] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:23 2022] DELETE /identity/v3/projects/8218c123052a4a72b928ec03f4b189cc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:23.720439 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:23.737513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75944c96-ec3d-449d-be20-5ba013549022 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:23.748507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:23.785403 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:23.786017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:23.786290 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:23.786560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] PATH_INFO: `/v3/users/cad7f3bcb9254cdbb82d6ab242d7bb42` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:23.787763 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=cad7f3bcb9254cdbb82d6ab242d7bb42)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:23.798232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:23.943830 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cad7f3bcb9254cdbb82d6ab242d7bb42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.956514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cad7f3bcb9254cdbb82d6ab242d7bb42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.957174 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cad7f3bcb9254cdbb82d6ab242d7bb42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:23.964137 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cad7f3bcb9254cdbb82d6ab242d7bb42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.033110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-83971eb9-966c-4951-a75e-b10a423292ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cad7f3bcb9254cdbb82d6ab242d7bb42'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.039019 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1430/2858] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:23 2022] DELETE /identity/v3/users/cad7f3bcb9254cdbb82d6ab242d7bb42 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 00:45:24.491753 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:24.493259 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:24.507183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e7656-8a86-4633-be55-eb3c661c61ba admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:24.564790 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:24.565340 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:24.565616 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:24.565886 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] PATH_INFO: `/v3/projects/6a436add251c44df82e43cb03d1b00bb` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:24.566991 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a436add251c44df82e43cb03d1b00bb)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:24.568912 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:24.646060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:24.665173 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a436add251c44df82e43cb03d1b00bb', 'request_id': 'req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.671973 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a436add251c44df82e43cb03d1b00bb', 'request_id': 'req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.678390 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a436add251c44df82e43cb03d1b00bb', 'request_id': 'req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.747085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] The token cache is being invalidate because project 6a436add251c44df82e43cb03d1b00bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:24.747426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.748071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-8ad3d129-2a8d-4db8-84a3-f9d0aa1576fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:24.749546 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1429/2859] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:24 2022] DELETE /identity/v3/projects/6a436add251c44df82e43cb03d1b00bb => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:24.755059 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:24.755555 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:24.755672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:24.776072 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:24.777279 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:24.835565 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1431/2860] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:24.844470 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:24.844786 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:24.845060 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:24.864523 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:24.865727 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:24.874008 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1430/2861] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:24.878110 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:24.888581 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c518f5-dc43-4d31-8348-9f32408e1467 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:24.893220 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:24.893817 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:24.894094 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:24.894365 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:24.895010 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:45:24.895342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:24.896934 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:24.908959 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1432/2862] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:45:24.913167 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:24.922606 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a5eed-92b8-4162-854a-af172b11bbd1 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:24.927299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:24.927907 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:24.928183 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:24.928452 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:24.929099 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:24.931005 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:25.029697 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1431/2863] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:45:24 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:25.033977 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:25.044371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d48bb-5cb9-46eb-a4c7-69879c948246 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:25.069761 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:25.101661 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:25.102263 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:25.102568 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:25.102839 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:25.103484 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:25.105428 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:25.176049 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1433/2864] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:45:25 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:45:25.180305 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:25.191083 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef3e15-58b1-4831-9326-ba66037b90a0 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:25.195776 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:25.196367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:25.196650 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:25.197653 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:25.198297 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:25.199904 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:25.209686 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1432/2865] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:25.214119 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:25.223816 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4dad34-dec7-4225-873f-cf785c63f4e9 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:25.228417 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feced3-beb3-4d0c-adc7-ff965785c9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:25.228992 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feced3-beb3-4d0c-adc7-ff965785c9cb admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:25.229270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feced3-beb3-4d0c-adc7-ff965785c9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:25.229541 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feced3-beb3-4d0c-adc7-ff965785c9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:25.230176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49feced3-beb3-4d0c-adc7-ff965785c9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:25.231812 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49feced3-beb3-4d0c-adc7-ff965785c9cb admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:25.241563 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1434/2866] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:25.246067 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:25.255820 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938a8d9-fd74-4f67-9806-1dc140e281cd admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:25.261085 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:25.261717 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:25.261997 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:25.262270 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] PATH_INFO: `/v3/projects/1d5ee00bc7374349bede6e4a02e23674/users/bf0a6b93ecdd4b4690bce1d9f15d2695/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:25.276613 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d5ee00bc7374349bede6e4a02e23674, user_id=bf0a6b93ecdd4b4690bce1d9f15d2695, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:25.279245 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:25.347745 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1433/2867] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:45:25 2022] PUT /identity/v3/projects/1d5ee00bc7374349bede6e4a02e23674/users/bf0a6b93ecdd4b4690bce1d9f15d2695/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:25.357953 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:25.358177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:25.358454 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:25.387266 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] MFA Rules not processed for user `bf0a6b93ecdd4b4690bce1d9f15d2695`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:25.388846 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:25.451556 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1435/2868] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:25.462423 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:25.464234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:25.520702 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7a2a-41a9-47f0-b827-2aa1cf766bc0 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:25.524472 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b66208-ab22-4ffd-9f18-6b10172f3870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:25.525071 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b66208-ab22-4ffd-9f18-6b10172f3870 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:25.525430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b66208-ab22-4ffd-9f18-6b10172f3870 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:25.525711 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b66208-ab22-4ffd-9f18-6b10172f3870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:25.533947 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b66208-ab22-4ffd-9f18-6b10172f3870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:25.535396 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b66208-ab22-4ffd-9f18-6b10172f3870 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:25.544744 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1434/2869] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:34.192766 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:34.254468 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38440a7b-3ffc-41f4-89a7-46c99fb86224 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:34.258919 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:34.259587 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:34.260422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:34.260688 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] PATH_INFO: `/v3/users/bf0a6b93ecdd4b4690bce1d9f15d2695` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:34.261852 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf0a6b93ecdd4b4690bce1d9f15d2695)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:34.263791 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:34.487926 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:34.493990 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf0a6b93ecdd4b4690bce1d9f15d2695'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:34.506295 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf0a6b93ecdd4b4690bce1d9f15d2695'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:34.506970 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf0a6b93ecdd4b4690bce1d9f15d2695'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:34.513746 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf0a6b93ecdd4b4690bce1d9f15d2695'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:34.576031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf0a6b93ecdd4b4690bce1d9f15d2695'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:34.582258 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1436/2870] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:34 2022] DELETE /identity/v3/users/bf0a6b93ecdd4b4690bce1d9f15d2695 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:34.588371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:34.588371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:34.588371 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:34.606861 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:34.608092 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:34.654024 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:34.665357 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1435/2871] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:34.675699 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:34.677103 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:34.737330 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e1418-8273-4329-9b55-4ef441a0f82a admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:34.741102 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4351e8-a79a-44d4-b48c-bf7da9527c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:34.741636 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4351e8-a79a-44d4-b48c-bf7da9527c65 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:34.741908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4351e8-a79a-44d4-b48c-bf7da9527c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:34.742180 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4351e8-a79a-44d4-b48c-bf7da9527c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:34.752149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4351e8-a79a-44d4-b48c-bf7da9527c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:34.753579 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4351e8-a79a-44d4-b48c-bf7da9527c65 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:34.763588 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1437/2872] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 00:45:35.120134 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.121585 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:35.136254 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b979073-fa98-4b64-a653-6129825fc5ab None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.140988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.141624 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.141902 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.142174 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] PATH_INFO: `/v3/projects/1d5ee00bc7374349bede6e4a02e23674` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.143430 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d5ee00bc7374349bede6e4a02e23674)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.145330 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:35.245535 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d5ee00bc7374349bede6e4a02e23674', 'request_id': 'req-9096e648-a12b-463f-80a5-a48296548c1f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:35.250989 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d5ee00bc7374349bede6e4a02e23674', 'request_id': 'req-9096e648-a12b-463f-80a5-a48296548c1f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:35.257143 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d5ee00bc7374349bede6e4a02e23674', 'request_id': 'req-9096e648-a12b-463f-80a5-a48296548c1f'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:35.329467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] The token cache is being invalidate because project 1d5ee00bc7374349bede6e4a02e23674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:35.329724 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:35.330393 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-9096e648-a12b-463f-80a5-a48296548c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:35.331906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1436/2873] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:35 2022] DELETE /identity/v3/projects/1d5ee00bc7374349bede6e4a02e23674 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:35.336773 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.337507 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.337640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.357482 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:35.358653 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:35.416825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1438/2874] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:45:35.479319 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.479605 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.479879 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.500367 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:35.501567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:35.509413 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1437/2875] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:35.514051 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.523660 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c2a38-063d-438b-ba3c-cf5b347d877f None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.528849 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.529435 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.529710 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.529978 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.530617 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:45:35.530917 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.532545 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:35.543829 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1439/2876] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:45:35.548137 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.557300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a06d43-5330-4de3-af6c-43a24e26d0f6 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.561835 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.562415 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.562687 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.562999 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.563637 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.565510 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:35.691798 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1438/2877] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:45:35 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:35.695957 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.706521 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79358a92-a451-4b45-9ad7-752aeb85e8c1 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.749391 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:35.762412 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.763038 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.763343 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.763614 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.764233 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.766176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:35.873672 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1440/2878] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:45:35 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 00:45:35.877888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.889338 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb314916-5a4a-417f-b292-5a6b69bd2038 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.893974 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.894646 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.894919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.895186 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.895845 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.897417 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:35.908171 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1439/2879] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:35.912149 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.922461 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ea015-a6f8-42b3-8df3-fdd80161cd49 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.927017 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774703a-d48a-41af-9b7f-868b8b48b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.927640 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774703a-d48a-41af-9b7f-868b8b48b680 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.927958 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774703a-d48a-41af-9b7f-868b8b48b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.928232 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774703a-d48a-41af-9b7f-868b8b48b680 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.928908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774703a-d48a-41af-9b7f-868b8b48b680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.930474 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774703a-d48a-41af-9b7f-868b8b48b680 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:35.942228 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1441/2880] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:35.946280 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:35.956939 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ab99a-2c03-4cba-9156-4501f5eab9f4 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:35.961514 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:35.962426 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:35.962712 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:35.963097 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] PATH_INFO: `/v3/projects/a8580b84ec7149f5b2a4d22df3db7d58/users/729a5eb78ae3425080771cdcac0789b7/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:35.977103 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8580b84ec7149f5b2a4d22df3db7d58, user_id=729a5eb78ae3425080771cdcac0789b7, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:35.980414 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:36.055906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1440/2881] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:45:35 2022] PUT /identity/v3/projects/a8580b84ec7149f5b2a4d22df3db7d58/users/729a5eb78ae3425080771cdcac0789b7/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:36.066372 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:36.066659 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:36.066960 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:36.094451 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] MFA Rules not processed for user `729a5eb78ae3425080771cdcac0789b7`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:36.095718 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:36.158321 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1442/2882] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:36.170059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:36.171888 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:36.227263 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef789-4273-4891-b77e-e15d952e852c admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:36.231469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd672a6-9ed9-4af4-b054-ee430dbd6106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:36.232059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd672a6-9ed9-4af4-b054-ee430dbd6106 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:36.232334 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd672a6-9ed9-4af4-b054-ee430dbd6106 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:36.232598 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd672a6-9ed9-4af4-b054-ee430dbd6106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:36.240915 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd672a6-9ed9-4af4-b054-ee430dbd6106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:36.242350 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd672a6-9ed9-4af4-b054-ee430dbd6106 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:36.252232 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1441/2883] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:40.067643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:40.128389 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17af5c-a797-41eb-b7c6-2096c280b613 None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:40.132918 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:40.133529 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:40.133805 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:40.134075 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] PATH_INFO: `/v3/users/729a5eb78ae3425080771cdcac0789b7` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:40.135808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=729a5eb78ae3425080771cdcac0789b7)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:40.137703 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:40.327413 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729a5eb78ae3425080771cdcac0789b7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:40.334224 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:40.339478 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729a5eb78ae3425080771cdcac0789b7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:40.340141 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729a5eb78ae3425080771cdcac0789b7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:40.346892 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729a5eb78ae3425080771cdcac0789b7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:40.461694 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729a5eb78ae3425080771cdcac0789b7'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:40.468312 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1443/2884] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:40 2022] DELETE /identity/v3/users/729a5eb78ae3425080771cdcac0789b7 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:40.472804 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:40.473295 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:40.473420 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:40.494416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:40.494657 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:40.505019 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:40.554130 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1442/2885] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 00:45:40.564470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:40.565882 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:40.622501 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441ff73-3f48-452c-8535-c1cb2eddd0ae admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:40.626161 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd915fb-4ed5-4bcc-bce0-89cdd5926587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:40.626728 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd915fb-4ed5-4bcc-bce0-89cdd5926587 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:40.627005 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd915fb-4ed5-4bcc-bce0-89cdd5926587 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:40.627305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd915fb-4ed5-4bcc-bce0-89cdd5926587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:40.636152 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd915fb-4ed5-4bcc-bce0-89cdd5926587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:40.637654 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd915fb-4ed5-4bcc-bce0-89cdd5926587 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:40.647123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1444/2886] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:41.005666 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.007248 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:41.019610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6159c6-c9f3-4bb7-9807-0540e021e070 None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.024397 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.025025 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.025300 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.025567 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] PATH_INFO: `/v3/projects/a8580b84ec7149f5b2a4d22df3db7d58` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.027460 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8580b84ec7149f5b2a4d22df3db7d58)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.029463 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.119531 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8580b84ec7149f5b2a4d22df3db7d58', 'request_id': 'req-2de06a41-8473-4880-b645-54e1ce686d67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:41.125213 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8580b84ec7149f5b2a4d22df3db7d58', 'request_id': 'req-2de06a41-8473-4880-b645-54e1ce686d67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:41.131299 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8580b84ec7149f5b2a4d22df3db7d58', 'request_id': 'req-2de06a41-8473-4880-b645-54e1ce686d67'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:41.192149 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] The token cache is being invalidate because project a8580b84ec7149f5b2a4d22df3db7d58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:41.192501 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:41.193122 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-2de06a41-8473-4880-b645-54e1ce686d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:41.194589 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1443/2887] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:41 2022] DELETE /identity/v3/projects/a8580b84ec7149f5b2a4d22df3db7d58 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:41.200015 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.200536 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.200656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.221047 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:41.222239 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:41.279560 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1445/2888] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:41.288418 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.288738 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.289012 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.308773 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:41.309965 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:41.317818 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1444/2889] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:41.321956 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.332422 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4bb0b-1f2d-4dcc-b8ed-543f87b02aba None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.337060 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.337927 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.338207 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.338480 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] PATH_INFO: `/v3/domains` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.339123 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91987) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 00:45:41.339464 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.341125 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.352505 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1446/2890] 10.222.0.15 () {58 vars in 1153 bytes} [Mon Aug 8 00:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 00:45:41.357264 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.366324 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef05795-5933-4d2a-b469-5a3274bb1e4b None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.371610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.372193 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.372467 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.372734 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.373348 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.375239 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.520421 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1445/2891] 10.222.0.15 () {60 vars in 1153 bytes} [Mon Aug 8 00:45:41 2022] POST /identity/v3/projects => generated 397 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:41.524585 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.534785 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adffbe4-fcfa-498f-bee0-1232c7ff177c admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.594455 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.595064 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.595370 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.595643 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] PATH_INFO: `/v3/users` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.596269 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.598176 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.608825 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:41.688607 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1447/2892] 10.222.0.15 () {60 vars in 1144 bytes} [Mon Aug 8 00:45:41 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 00:45:41.692544 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.700301 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b294771-ff68-435a-aafc-b936e3800f13 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.705148 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.707913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.707913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.707913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.707913 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.708536 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.719318 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1446/2893] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:41.723376 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.734486 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee63454-7f75-4b32-8373-0e57219a5aac admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.739140 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c99bf2-ea2c-44d7-8787-8a1574f14314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.739759 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c99bf2-ea2c-44d7-8787-8a1574f14314 admin admin] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.740045 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c99bf2-ea2c-44d7-8787-8a1574f14314 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.740305 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c99bf2-ea2c-44d7-8787-8a1574f14314 admin admin] PATH_INFO: `/v3/roles` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.740939 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c99bf2-ea2c-44d7-8787-8a1574f14314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.742514 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c99bf2-ea2c-44d7-8787-8a1574f14314 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.753506 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1448/2894] 10.222.0.15 () {58 vars in 1122 bytes} [Mon Aug 8 00:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 00:45:41.757663 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:41.767908 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543e37-2655-4193-8d8a-aab9ad090f56 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:41.772722 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:41.773286 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.773565 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.773839 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] PATH_INFO: `/v3/projects/0b446883fd7a4c7c861362b8def5b938/users/df0b1740e7b54ed6ab2058ce9e055158/roles/094f7bfb155b40fea515d8a0935ab3db` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.788815 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b446883fd7a4c7c861362b8def5b938, user_id=df0b1740e7b54ed6ab2058ce9e055158, role_id=094f7bfb155b40fea515d8a0935ab3db)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:41.791629 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:41.890317 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1447/2895] 10.222.0.15 () {60 vars in 1483 bytes} [Mon Aug 8 00:45:41 2022] PUT /identity/v3/projects/0b446883fd7a4c7c861362b8def5b938/users/df0b1740e7b54ed6ab2058ce9e055158/roles/094f7bfb155b40fea515d8a0935ab3db => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:41.905553 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] REQUEST_METHOD: `POST` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:41.905869 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:41.906148 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:41.934559 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.auth.core [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] MFA Rules not processed for user `df0b1740e7b54ed6ab2058ce9e055158`. Rule list: `[]` (Enabled: `True`). {{(pid=91987) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:41.935808 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:41.998656 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1449/2896] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:42.009764 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:42.012192 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:42.052258 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:42.070291 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719516b-c132-48ae-b695-223d78be2cd7 admin admin] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:42.074566 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a251cb-1dfe-4e3e-8e58-dc0d27fcc065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e66698136f4429b8df67e5050a8da2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:42.075161 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a251cb-1dfe-4e3e-8e58-dc0d27fcc065 service cinder] REQUEST_METHOD: `GET` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:42.075469 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a251cb-1dfe-4e3e-8e58-dc0d27fcc065 service cinder] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:42.075739 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a251cb-1dfe-4e3e-8e58-dc0d27fcc065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:42.084374 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a251cb-1dfe-4e3e-8e58-dc0d27fcc065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:42.085914 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a251cb-1dfe-4e3e-8e58-dc0d27fcc065 service cinder] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:42.096234 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1448/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:50.832050 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:50.896379 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd575b-2437-40ce-9c04-79d525308a9c None None] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:50.900908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:50.901528 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:50.901801 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:50.902068 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] PATH_INFO: `/v3/users/df0b1740e7b54ed6ab2058ce9e055158` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:50.903270 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0b1740e7b54ed6ab2058ce9e055158)` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:50.909096 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:51.091706 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0b1740e7b54ed6ab2058ce9e055158'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.103470 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0b1740e7b54ed6ab2058ce9e055158'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.104129 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0b1740e7b54ed6ab2058ce9e055158'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.111371 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0b1740e7b54ed6ab2058ce9e055158'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.113025 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:45:51.164041 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.notifications [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0b1740e7b54ed6ab2058ce9e055158'} {{(pid=91987) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.171334 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1450/2898] 10.222.0.15 () {58 vars in 1224 bytes} [Mon Aug 8 00:45:50 2022] DELETE /identity/v3/users/df0b1740e7b54ed6ab2058ce9e055158 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:45:51.175988 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] REQUEST_METHOD: `POST` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:51.181809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:51.181809 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:51.196079 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.auth.core [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] MFA Rules not processed for user `de917b8f456944dc9d27423048f81375`. Rule list: `[]` (Enabled: `True`). {{(pid=91988) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 00:45:51.200416 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:51.257102 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1449/2899] 10.222.0.15 () {58 vars in 958 bytes} [Mon Aug 8 00:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 00:45:51.267328 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:51.269342 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:51.326784 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443918a-f473-4e99-9edd-6b6a82f09b61 admin admin] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:51.330481 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c8a8736fc89485d87f14e06e85e67c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:51.331081 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:51.331392 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:51.331668 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:51.341513 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:51.342932 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:51.352336 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1451/2900] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:45:51.731919 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] Authenticating user token {{(pid=91988) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:45:51.733604 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.fernet_utils [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91988) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:45:51.749933 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5392-12d2-46d7-99a9-0a92745f99ff None None] Validating token access rules against request {{(pid=91988) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:45:51.751906 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de917b8f456944dc9d27423048f81375', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '311e504f385e4c32bdb056e763a01299', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91988) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:45:51.752538 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:45:51.752811 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:45:51.753076 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] PATH_INFO: `/v3/projects/0b446883fd7a4c7c861362b8def5b938` {{(pid=91988) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:45:51.754284 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b446883fd7a4c7c861362b8def5b938)` {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:45:51.756983 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] RBAC: Authorization granted {{(pid=91988) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:45:51.862059 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b446883fd7a4c7c861362b8def5b938', 'request_id': 'req-b8db403d-e5c7-4bb5-9940-305a86768aa9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.867871 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b446883fd7a4c7c861362b8def5b938', 'request_id': 'req-b8db403d-e5c7-4bb5-9940-305a86768aa9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.874088 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b446883fd7a4c7c861362b8def5b938', 'request_id': 'req-b8db403d-e5c7-4bb5-9940-305a86768aa9'} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.923180 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] The token cache is being invalidate because project 0b446883fd7a4c7c861362b8def5b938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91988) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 00:45:51.923512 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.924136 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG keystone.notifications [None req-b8db403d-e5c7-4bb5-9940-305a86768aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91988) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 00:45:51.925610 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: [pid: 91988|app: 0|req: 1450/2901] 10.222.0.15 () {58 vars in 1233 bytes} [Mon Aug 8 00:45:51 2022] DELETE /identity/v3/projects/0b446883fd7a4c7c861362b8def5b938 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 00:46:01.348177 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=28 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:46:01.920230 dv-cinder-851640-3-iscsi devstack@keystone.service[91988]: DEBUG dbcounter [-] [91988] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91988) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 00:46:42.593235 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] Authenticating user token {{(pid=91987) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 00:46:42.594631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:46:42.662584 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f6d99-d255-4ee2-9c65-94a67a25e7d8 service neutron] Validating token access rules against request {{(pid=91987) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 00:46:42.720860 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21c1437c38d743c585ce98f7a3e96627', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d7e3c85c1c141d1965b1c8f0a5cbe21', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91987) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 00:46:42.721572 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] REQUEST_METHOD: `GET` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 00:46:42.721908 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] SCRIPT_NAME: `/identity` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 00:46:42.722179 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91987) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 00:46:42.724031 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.fernet_utils [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91987) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 00:46:42.735011 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 00:46:42.736631 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b4297-c4d5-466f-89a5-5a7073f0fec4 service placement] RBAC: Authorization granted {{(pid=91987) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 00:46:42.747126 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: [pid: 91987|app: 0|req: 1452/2902] 10.222.0.15 () {60 vars in 1365 bytes} [Mon Aug 8 00:46:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 00:46:52.742864 dv-cinder-851640-3-iscsi devstack@keystone.service[91987]: DEBUG dbcounter [-] [91987] Writing DB stats keystone:SELECT=58 {{(pid=91987) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}